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Abstract 


Many  planning  tasks  can  be  represented  using  mental  models  in  which  an  expert 
manipulates  objects  from  one  state  to  another  (delivery  route  planning  -  trucks,  buildings, 
packages,  routes,  etc.;  part  machining  -  parts,  drill,  mill,  drill-bit,  etc.).  This  suggests  a 
highly  graphical  knowledge  acquisition  tool  where  the  expert  is  able  to  capture  the  visual 
intuition  of  die  problem  solving  to  facilitate  the  encoding  of  a  domain  knowledge  base.  By 
exploring  knowledge  acquisition  for  object  manipulation  domains,  insight  will  be  gained  in 
how  knowledge  is  acquired  and  represented  for  such  visually  oriented  tasks. 

This  thesis  addresses  graphical  knowledge  acquisition  in  visually  oriented  domains  in  the 
context  of  Prodigy,  a  general  problem  solving  and  planning  architecture.  The  prototype 
system,  called  APPRENTICE,  demonstrates  the  main  ideas  in  the  thesis.  This  system 
establishes  the  feasibility  of  a  graphical  interface  to  enhance  the  ability  of  the  expert  to 
develop  factual  domain  knowledge  (objects,  relations,  and  operators)  in  multiple  domains. 

The  system  has  been  evaluated  in  four  studies.  In  the  first  study,  32  AI  students  used  the 
system  to  build  their  own  domains.  In  the  second  study,  domains  developed  by  different 
types  of  users  were  completed  faster  using  graphical  input  than  using  textual  input.  The 
third  study  was  a  learning  study  in  which  a  subject  developed  several  domains  in 
APPRENTICE.  Finally,  the  fourth  study  demonstrated  the  ability  to  develop  a  larger 
domain  in  the  system.  APPRENTICE  and  its  techniques  proved  to  be  usable,  flexible  and 
extendable. 
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Chapter  1  •  Motivation 


1.1  Preview  of  Thesis 

This  thesis  investigates  the  process  of  producing  and  exploiting  graphically  based 
gpwrrifirafinns  of  visual  domains  for  a  general  purpose  planning  system.  The  central  thesis 
is  fha*  graphical  specification  helps  both  to  reduce  domain  development  time  and  to 
improve,  the  accuracy  of  knowledge  capture.  The  knowledge  acquisition  (KA)  problem  is 
presented  and  discussed  in  section  1.2. 


This  thesis  sets  out  to: 

•  address  the  problems  of  ease,  speed,  and  accuracy  of  knowledge 
acquisition  for  highly  visual  planning  domains; 

•  develop  techniques  for  providing  knowledge  acquisition  access  to 
people  with  domain  expertise  but  with  no  knowledge  acquisition 
skills; 

•  validate  empirically  these  techniques  across  multiple  domains  and 
with  different  user  populations; 

•  incorporate  new  graphical  knowledge  acquisition  methods  into  a 
general  purpose  planning  system  without  limiting  its  expressiveness. 


A  general  paradigm  has  been  developed  for  graphically  defining  a  broad  range  of 
planning  domains  that  Tend  themselves  to  a  visual  representation.  A  system,  called 
APPRENTICE,  has  been  developed,  tested,  and  used  to  build  many  visual  domains.  Four 
studies  were  done  with  APPRENTICE  to  evaluate  its  performance  with  multiple  users  and 
numerous  domains,  as  well  as  the  evolution  of  the  system's  use  over  time.  These  studies 
suggest  that  the  APPRENTICE  techniques  help  the  user  to  decrease  development  time, 
increase  debugging  efficiency,  and  comprehend  existing  domains  faster. 

1J.1  Example  of  a  Domain  Being  Developed  in  APPRENTICE 
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The  APPRENTICE  system  tries  to  mimic  some  of  the  techniques  that  experts  use  to 
convey  domain  knowledge  to  a  student  or  an  apprentice.  When  teaching  a  new  domain  the 
expert  first  tries  to  systematically  teach  the  student  the  objects,  attributes,  and  relations  in 
the  domain,  thus  establishing  a  common  vocabulary.  The  expert  and  student  can  then 
comfortably  discuss  this  area  of  expertise.  This  technique  can  be  used  to  develop  many 
different  domains;  but  to  illustrate  tins  interaction,  I  will  describe  the  development  of  a 
paction  of  a  machining  task.  This  task  is  to  d'-velop  a  simple  set  of  rules  for  drilling  a  hole 
in  a  part  In  Chapter  4,  this  domain  will  be  expanded  into  a  larger  one. 

When  experts  describe  a  domain  they  first  start  by  defining  the  objects  that  exist  in  that 
domain.  Figure  1.1  shows  different  objects  in  the  small  machining  domain.  In  this 
example  the  expert  uses  these  objects  to  talk  about  the  domain. 


I 

Dtifl-bit  Hois 
Figure  LI:  Several  objects  that  are  in  a  machining  domain. 

The  pictures,  along  with  the  names  of  the  objects,  give  the  student  a  physical 
representation.  The  drill,  drill  bit,  vise,  part,  and  hole  are  now  part  of  the  common 
vocabulary  between  expert  and  student 


Once  a  subset  of  die  objects  is  defined,  the  expert  can  describe  how  these  objects  relate  to 
one  another.  In  Figure  1.2  the  relations  between  the  objects  in  our  machining  domain  are 


Vi*  o««  Drill  ViMhotfagnltot  Hole  ia  Put  DrilMnt  in  chill 


Figure  L2:  Some  relations  between  objects  in  the  machining  domain. 
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These  relations  allow  the  expert  to  discuss  how  objects  interact  with  one  another.  Again, 
the  pictures  and  text  allow  the  student  to  fonn  a  visual  representation  of  relations.  These  are 
similar  to  the  way  the  expert  thinks  about  the  object  interactions. 

With  the  objects  and  relations  defined,  the  expert  can  then  discuss  conjunctive  sets  of 
relations  or  states.  Figure  1.3  depicts  a  state  in  the  machining  domain.  This  figure 
represents  the  set  of  relations:  vis*  on  a  drill,  vise  holding  a  part,  and  drill- 
bit  in  the  drill. 


Figure  L3:  State  in  the  machining  domain. 

This  representation  makes  it  relatively  easy  to  visualize  what  is  actually  done  in  a 
machining  shop.  The  expert  is  now  able  to  discuss  the  machining  domain  based  upon  the 
expert's  knowledge  developed  over  time. 

The  expert  can  now  discuss  operations  that,  when  applied,  change  the  state.  To  describe 
these  operators  the  expert  first  defines  a  pre-state  (a  state  that  has  to  exist  before  the 
operator  can  be  applied),  and  then  a  post-state  (a  state  that  exists  after  the  operator  has  been 
applied).  In  Figure  1.4  a  simple  operator,  put -part -in-vise,  is  developed.  This  operator 
depicts  the  process  of  putting  a  part  into  a  vise. 


Figure  L4:  The  put -part -in- vise  operator  in  the  machining  domain. 
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The  pre-state  defines  the  conditions  which  are  necessary  before  die  operator  can  be  applied: 
the  vise  must  be  on  the  drill  and  the  drill  bit  must  be  in  the  drill.  If  these  relations  are  true  in 
the  state  then  the  operator  can  be  applied  and  the  state  changed  to  reflect  that  the  part  is  in 
the  vise.  These  pie-  and  post-states  represent  the  preconditions  and  the  state  transition  for 
the  put -part -in- vise  operator.  Figure  1.5  shows  some  other  operators  that  are  defined 
in  a  similar  way  (put -drill-bit-in-drill,  put-vise-on-drill,  drill-hole-in¬ 
part). 


Drill-hole- in-part 

Figure  L5:  Other  operators  in  the  machining  domain. 


Finally,  this  information  can  be  used  by  a  planner  to  produce  a  sequence  of  operators  that, 
when  applied,  will  transform  an  initial  state  into  a  final  state.  The  planner  does  this  by  first 
determining  which  operators  achieve  a  goal  and,  if  they  are  not  applicable,  which  operators 
establish  the  pre-state  of  the  previously  related  operator.  This  continues  recursively  until  a 
sequence  of  operators  is  found  that  is  both  applicable  to  the  initial  state  and  achieves  the 
goaL  This  process,  called  Means-Ends  Analysis  [Newell  72]  may  mean  that  the  system 
has  to  try  several  alternative  operator  applications.  Figure  1.6  illustrates  one  set  of  state 
transitions  that  the  planning  system  took  to  solve  the  machining  domain  problem  earlier 
defined. 
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Figure  L6:  Sequence  of  operators  in  the  machining  domain  to  drill  a  hole  in  a  part 

Each  square  represents  the  current  state  of  the  problem  after  the  operator  under  die  picture 
has  been  applied.  Note  that  the  initial  state  is  composed  of  a  part,  a  drill  bit  a  vise  and  the 
drill  unassembled;  the  goal  state  is  to  have  the  part  with  a  hole  in  it  The  sequence  of 
operators  determined  in  Figure  1.6  to  transform  the  initial  state  into  the  goal  state  is: 

1)  put -vise-on-drill 

2)  put -drill-bit-in-vise 

3 )  put -part - in-vise 

4)  drill-hole-in-part 

As  the  planner  tries  to  find  a  solution,  the  expert  can  monitor  the  planner's  progress.  By 
monitoring  the  planner  the  expert  can  detect  erroneous  or  incomplete  knowledge.  This 
helps  with  debugging  domains,  which  is  a  significant  part  of  domain  development  The 
expert  can  also  verify  a  sequence  of  operators  by  applying  the  operators  (me  at  a  time. 

APPRENTICE  allows  the  expert  to  specify  domain  information  similar  to  the  domain 
description  previously  outlined.  APPRENTICE,  of  course,  handles  significantly  more 
complex  domains  as  well.  Chapter  3  describes  this  process  in  detail. 
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12  Problem  Outline 

Expert  systems  are  increasingly  being  used  in  business  and  industry.  The  development  of 
these  systems  requires  the  acquisition  of  knowledge  from  experts,  a  very  time-consuming 
process  for  both  domain  experts  and  knowledge  engineers.  This  knowledge  acquisition 
(KA)  process  has  been  characterized  as  "the  transfer  and  transformation  of  problem¬ 
solving  expertise  from  some  knowledge  source  to  a  program"  [Buchanan  83].  By  making 
the  KA  process  foster  and  easier  for  experts,  the  domain  development  time  and  expense 
will  be  reduced. 


Machine  learning  methods  are  helping  to  automate  knowledge  base  development  and 
improvement  [Minton  88].  Nonetheless,  initial  domain  development  has  heretofore  been 
manual,  and  humans  are  still  needed  for  domain  refinements.  Developing  domains  can  be 
very  difficult  and  time  consuming,  especially  when  the  development  tool  does  not  reflect  a 
simple  mapping  from  the  external  domain  to  the  system's  user  interface. 


Figure  1.7:  Diagram  of  typical  knowledge  acquisition  process. 


Knowledge  base  development  is  an  interactive  process.  This  is  depicted  in  Figure  1.7.  The 
role  of  the  expert  is  to  impart  domain  knowledge  for  the  knowledge  engineer  to  encode 
into  die  knowledge  base.  The  knowledge  engineer  and  die  expert  may  be  the  same  person 
or  different  people  and  they  work  towards  developing  the  knowledge  base.  The  knowledge 
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engineer  has  the  responsibility  of  translating  the  expertknowledge  into  a  representation  for 
the  export  system.  If  this  representation  is  very  different  from  the  expert’s  representation 
then  there  are  opportunities  to  introduce  and  propagate  errors.  These  errors  and 
misconceptions  required  diagnosis  and  correction,  and  thereby  add  to  the  development  time 
afadomain. 

In  order  to  make  this  human  development  process  more  efficient,  tools  are  needed  to 
facilitate  the  initial  domain  specification,  to  aid  in  finding  erroneous  or  incomplete 
information  in  a  domain,  and  to  help  in  correcting  that  domain  information.  This  should  be 
done  within  a  paradigm  that  represents  knowledge  in  a  way  experts  can  use  and 
understand.  This  thesis  examines  a  restricted  type  of  knowledge  acquisition  that  non¬ 
computer  experts  can  use.  I  focus  on  the  human  interactive  process  involved  when  an 
expert  develops  a  knowledge  base  for  a  planning  system  where  the  domain  is  composed  of 
physical  or  mental  objects  being  manipulated. 

A  graphical  knowledge  acquisition  tool  aids  in  domain  development  The  expert 
communicates  with  the  computer  in  a  way  similar  to  the  way  an  expert  understands  a 
domain,  and  the  computer  is  responsible  for  transforming  die  information  into  a  form  that 
it  can  use.  Also,  having  the  computer  communicate  back  to  the  expert  in  terms  that  the 
expert  is  more  familiar  with  better  enables  the  expert  to  debug  and  enhance  the  system's 
knowledge. 

By  focusing  on  the  acquisition  of  knowledge  for  visual  planning  tasks  in  an 
expert/apprentice  interaction,  I  show  how  knowledge  base  development  can  be  facilitated 
graphically  and  improved  for  a  range  of  users,  including  domain-knowledgeable  but 
system-novice  users. 

13  Significance  of  Research 

The  research  in  this  dissertation  makes  four  significant  contributions: 

•  a  new  knowledge  acquisition  methodology  is  developed  for  visual  domains  to 
decrease  development  time, 

•  the  techniques  can  be  applied  by  users  without  knowledge  acquisition  skills, 

•  die  methodology  is  implemented  and  tightly  integrated  with  a  planning  system 
(Prodigy), 
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•  the  methodology  is  tested  by  building  multiple  domains  and  using  multiple  subjects 
with  varying  skills. 

The  new  graphical  approach  to  knowledge  acquisition  allows  the  expert  to  reason  about 
domain  building  in  a  way  similar  to  how  the  expert  thinks  about  the  domain.  These  general 
techniques  can  be  separated  into  basic  elements  for  developing  domains  for  a  general 
expert  system.  With  these  techniques,  the  expert  communicates  with  the  system  in  a  way 
that  is  comfortable  to  the  expert;  therefore,  knowledge  acquisition  takes  less  time  to 
complete.  Also,  die  domain  development  can  be  done  by  experts  that  are  not  familiar  with 
planning  systems.  This  is  radically  different  from  other  knowledge  acquisition  systems 
which  provide  detailed  insight  into  the  expert  system,  but  in  a  way  foreign  to  the  actual 
domain.  In  such  systems  the  expert  or  knowledge  engineer  is  responsible  for  translating 
domain  knowledge  into  the  representation  that  the  system  expects. 

The  techniques  developed  in  this  thesis  are  used  in  building  a  working  system  that  is  tightly 
coupled  with  a  planning  system.  This  development  allows  proof  of  concept,  testing  of 
ideas,  and  enhancement  of  the  methodology.  Also,  unlike  knowledge  acquisition  tools  that 
require  a  separate  program  to  run  the  domain,  APPRENTICE  allows  interactive  domain 
development  within  a  planning  system.  The  system  is  tied  directly  with  the  planning 
system  so  that  the  results  of  a  developed  domain  can  be  run  on  the  system  instantaneously. 
This  is  similar,  to  using  an  interpretive  language  like  LISP  for  development 

Once  the  system  was  developed,  several  experiments  were  designed  and  conducted  to 
determine  whether  the  ideas  postulated  in  die  thesis  were  true.  These  studies  were  done 
with  different  types  of  users  doing  different  tasks.  Results  from  these  studies  showed  that 
different  types  of  people  were  able  to  understand,  build,  and  debug  domains.  The 
productivity  of  subjects  was  shown  to  be  higher,  and  the  subjects  also  found 
APPRENTICE  more  understandable  than  conventional  methods. 

1.4  Results 

The  primary  contribution  of  these  studies  is  to  demonstrate  the  range  of  application  and  the 
efficiency  of  APPRENTICE.  Most  domains  were  acquired  fully  with  the  graphical 
interface,  and  acquisition  speed  proved  faster  for  all  types  of  users  except  Prodigy  experts. 
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To  investigate  the  claim  of  die  thesis,  four  studies  were  performed.  These  studies  tested  the 
versatility,  flexibility,  and  usability  of  the  APPRENTICE  system  with  multiple  subjects  in 
die  development  of  several  domains. 

In  the  first  study,  APPRENTICE  was  used  to  build  a  diverse  set  of  user-defined  domains 
by  32  novice  users.  The  users  built  domains  of  their  own  choosing.  It  took  each  user 
between  one  and  4.5  hours  to  construct  a  simple  domain.  All  users  succeeded  in  building 
functional  domains,  thereby  proving  the  breadth,  usability,  and  versatility  of 
APPRENTICE. 

In  the  second  study,  the  APPRENTICE  system  yielded  faster  development  time  than 
conventional  methods  for  several  types  of  users.  This  was  a  comparison  study  with  four 
sets  of  subjects  with  varying  computer  skills.  Their  skills  ranged  from  novice  computer 
users  to  advance  planner  experts.  The  development  time  of  building  a  domain  in 
APPRENTICE  was  compared  with  the  development  time  using  a  text  editor  (Emacs).  All 
but  the  most  seasoned  planner  experts  built  domains  significantly  faster  in  APPRENTICE 
than  in  Emacs.  Another  phase  of  this  study  tested  the  ability  of  subjects  to  understand 
domains  developed  by  others.  This  ability  is  important  in  domain  development  and  system 
maintenance.  The  subjects  understood  the  graphical  representation  of  a  domain  more 
accurately  (i.e.,  they  answered  a  set  of  questions  about  the  domain  more  correctly)  than 
they  did  far  a  comparable  textual  representation.  This  study  proved  die  effectiveness  of 
APPRENTICE  in  actual  usage  and  validated  the  graphical  KA  paradigm  for  some 
important  domains. 

In  the  third  study,  APPRENTICE  was  used  by  one  user  to  build  a  medium  size  domain 
that  had  33  objects,  77  relations,  and  34  operators  in  it  For  this  domain  the  graphical 
representation  helped  to  enrich  the  development  process.  The  pictures  acted  as  visual  cues 
in  helping  die  subject  develop  the  domain.  For  example,  it  was  easy  to  recognize  and 
correct  the  situation  that  a  drill  did  not  have  a  drill  bit  when  creating  the  drill  operator. 
Another  advantage  was  that  the  graphics  helped  the  user  to  remember  the  context  in  which 
he  was  working  when  he  came  back  to  the  work  after  an  interruption. 

Finally,  die  last  study  was  done  with  one  user  building  several  similar  domains  over  time, 
hi  this  study  the  user  took  longer  to  build  the  first  domain  but  less  time  to  build  subsequent 
domains.  This  indicates  that,  as  users  build  domains  in  APPRENTICE,  their  productivity 
increases. 
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These  studies  provide  an  indication  of  the  merits  of  the  APPRENTICE  system.  The 
system  has  been  used: 

•  by  multiple  users 

•  to  developed  multiple  domains 

•  over  a  period  of  time 

•  to  developed  a  medium  size  domain. 

Overall  APPRENTICE  was  successfully  used  by  41  different  people  and  41  distinct 
domain*  were  developed.  These  study  results  help  to  support  the  thesis  of  this  dissertation. 

1.5  Reader's  Guide 

Chapter  2  compares  the  research  of  this  thesis  with  other,  related  work.  Chapter  3  describes 
the  implementation  of  the  system.  In  Chapter  4  the  empirical  studies  are  explained  and  the 
results  shown.  Chapter  5  summarizes  what  I  learned,  and  Chapter  6  draws  some 
conclusions.  The  Appendices  contain  supporting  material  and  are  not  needed  in  the 
understanding  of  die  concepts  reported  in  this  thesis.  Nevertheless,  the  information  they 
contain  may  be  of  interest  to  some  people,  such  as  those  attempting  to  replicate  or  expand 
upon  the  APPRENTICE  system. 
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Chapter  2  -  Related  Work 


The  design  and  implementation  of  APPRENTICE  draws  upon  ideas  from  several 
disciplines:  user  interfaces;  visual  programming  and  visual  languages;  machine  learning; 
and,  the  most  obvious,  knowledge  acquisition.  The  first  three  disciplines  complement  my 
work  and  have  provided  fertile  ground  from  which  to  borrow  many  techniques.  I  will 
discuss  some  of  the  techniques  that  I  used.  The  last  discipline,  knowledge  acquisition  has 
also  provided  many  advantages  to  my  work  and  helped  me  to  explore  new  ideas.  I  will 
contrast  my  work  with  others  in  the  area  of  knowledge  acquisition. 

User  Interface 

Techniques  from  the  field  of  user  interfaces  were  used  to  develop  the  human  computer 
interaction  for  the  APPRENTICE  system.  Direct  manipulation  was  the  basis  for  the 
interface  design.  This  gives  users,  both  novice  and  expert,  an  intuitive  feel  for  how  to  use 
the  system.  User  actions  cause  immediate  reactions  which  are  easy  to  relate  to. 

With  the  user  interface  several  different  types  of  interfaces  were  used.  Techniques 
supported  by  MacDraw  [Mac Draw  89]  were  adapted  to  develop  the  user-defined  objects. 
This  provides  users  with  an  easily  understood  interface  for  creating  simple  graphical 
objects.  Icon  manipulation  enables  production  of  other  domain  elements  with  these  objects 
as  building  blocks.  This  allows  users  shortcuts  for  developing  relations  between  the  objects 
represented  as  icons  [Macintosh  87].  The  interface  also  strives  for  consistency,  user 
feedback,  handling  of  user  mistakes,  a  simple  display  and  actions  that  have  quick  feedback. 
These  are  important  qualities  of  a  good  interface  design.  These  and  other  techniques  are 
described  in  much  of  the  literature  [Goldberg  80]  [Shneiderman  86]  [Cardelli  88]  [Myers 
92]. 


Visual  Programming  and  Visual  Languages 

Visual  programming  and  visual  languages  are  orthogonal  fields  of  study  that  benefited 
APPRENTICE.  The  goals  of  Visual  Programming  are  to  create  working  programs  [Cox 
89],  systems  [Ichikawa  87]  [Fischer  88]  or  investigate  ideas  [Henderson  86]  [Gutfreund 
87].  These  are  different  from  the  goal  of  APPRENTICE,  which  is  to  develop  domains  for 
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a  planning  system.  One  difference  in  the  way  that  APPRENTICE  works  is  that  most 
visual  programming  systems  have  a  predefined  set  of  icons,  and  therefore  the  users  have  to 
conform  their  thinking  about  the  system  in  the  way  that  the  software  designers  had  in 
mind.  To  aid  in  domain  development,  APPRENTICE  provides  the  user  with  the  ability  to 
define  the  objects  of  the  domain  in  a  visual  representation  that  corresponds  closely  to  the 
physical  world.  This  allows  the  user  a  comfortable  environment  to  work  in,  where  ideas 
representing  physical  relations  do  not  have  to  be  transformed  into  an  unrelated 
representation. 

The  advantages  of  animation  have  been  demonstrated  by  systems  such  as  BALSA-I 
[Brown  84],  BALSA-II  [Brown  88],  Animus  [Duisberg  86],  and  STEAMER  [Hollan 
84].  Animation  is  also  used  in  APPRENTICE  to  give  the  user  feedback  when  the  system 
is  solving  a  problem. 

Machine  Learning 

Machine  learning  is  another  field  of  study  which  has  had  influence  on  my  work.  Currently, 
machine  learning  research  cannot  create  initial  domain  information  without  the  aid  of  a 
human.  Automated  learning  methods  allow  knowledge  that  has  already  been  encoded  to  be 
refined.  This  can  be  achieved  by  decreasing  the  search  space  as  with  EBL  [Minton  88],  by 
statically  compiling  knowledge  [Entzio  90],  by  using  abstraction  techniques  [Knoblock  90] 
or  analogous  plan  usage  [Veloso  92]  or  by  adding  information  to  an  already  existing 
knowledge  base  as  with  experimentation  [Gil  92].  APPRENTICE  is  a  tool  that  helps  with 
initial  domain  formulation,  and  therefore  could  be  a  useful  complement  to  machine 
learning  systems. 

Also,  search  control  rules  [Minton  89]  (which  can  be  learned  by  different  methods  [Minton 
88]  [Entzio  90])  could  be  directly  used  in  APPRENTICE  to  decrease  search  time. 
Currently,  no  matter  how  these  search  control  rules  are  obtained  they  can  be  used  with 
APPRENTICE. 

Knowledge  Acquisition 

In  the  knowledge  acquisition  field  several  techniques  have  been  used  to  try  to  elicit  expen 
knowledge,  and  map  that  knowledge  into  expen  system  primitives. 
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Alexander  cl  al.  [Alexander  87]  have  developed  a  technique  called  ontological  analysis  or 
SUPE-SPOONS  which  aids  in  knowledge-level  analysis  of  a  problem  space.  The  analysis 
divides  domain  knowledge  into  three  categories: 

1)  Static  Ontology  •  physical  objects  or  primitive  objects  in  a  problem  space,  their 
properties  and  relations. 

2)  Dynamic  Ontology  -  state  space  of  the  problem-solving  domain  and  the  actions  that 
transform  the  problem  from  one  state  to  another  state. 

3)  Epistemic  Ontology  -  constraints  and  methods  that  control  the  use  of  knowledge 
applied  to  the  static  and  dynamic  ontologies. 

Complex  domain  ontologies  are  constructed  in  a  three-step  process  in  the  order  that  the 
categories  are  presented.  The  system  ASTEK  [Jacobson-90],  based  on  the  SUPE- 
SPOONS  research,  provides  multiple  paradigms  for  knowledge  editing  while  maintaining 
a  single  consistent  framework.  This  knowledge  editing  is  in  the  form  of  natural  language 
and  graph  editing  techniques. 

The  ontologies  created  with  the  system  are  very  similar  to  the  workings  of  APPRENTICE. 
The  static  ontology  parallels  object  and  relation  definitions.  The  dynamic  ontology  is  like 
the  operator  definition.  The  epistemic  ontology  is  similar  to  search  control  rules.  Like  the 
ontological  analysis  technique,  APPRENTICE  builds  its  knowledge  base  in  similar  stages. 
APPRENTICE  attempts  to  capture  and  exploit  both  the  visual  and  semantic  domain 
knowledge. 

Systems  that  take  different  approaches  include:  KREME  [Abrett  87]  a  knowledge 
representation  editing  and  modeling  environment;  KRITON  [Diederich  90]  a  hybrid 
system  that  uses  interview,  protocol  analysis,  and  incremental  text  analysis;  and 
AQUINAS  [Boose  87]  a  system  that  interviews  experts  to  build  tables  of  distinctions  or 
repertory  grids  between  elements.  The  systems  fall  into  the  following  information 
gathering  techniques: 

•  Visual  and  Semantic  Acquisition  (APPRENTICE) 

•  Interviewing  and  Protocol  (KRITON) 

•  Clustering  or  Scaling  (AQUINAS) 

•  Natural  Language  Systems  (ASTECK ,  KRITON) 

•  Visual  Graphs  of  Underlying  Data  Structure  (KREME) 


Page  13 


PROTEGE  [Musen  88]  has  had  success  providing  a  graphical  program  for  developing 
graphical  knowledge  acquisition  and  editing  tools.  These  knowledge  acquisition  tools  elicit 
knowledge  for  the  domain  independent  planner  e-ONCOCIN,  which  uses  the  method  of 
skeletal  refinement  for  problem  solving.  PROTEGE  has  been  used  to  create  KA  tools  for 
oncology  protocols  (p-OPAL)  and  hypertension  protocols  (HTN).  With  the  KA  tools, 
doctors,  without  the  aid  of  knowledge  engineers,  can  create  expert  systems  to  advise  in  the 
treatment  of  oncology  and  hypertension.  The  interface  to  these  knowledge  acquisition  tools 
uses  form  filling  and  graphical  flow  chart  building  to  elicit  the  domain  knowledge.  Unlike 
APPRENTICE,  PROTEGE  is  not  integrated  with  the  target  expert  system.  Also, 
PROTEGE’S  domain  is  more  limited,  although  the  authors  are  currently  expanding  the 
system  into  PROTEGE-II.  Finally  another  important  difference  is  that  PROTEGE  is  not  a 
system  for  developing  planning  domains. 

QMR-KAT  [Giuse  90]  is  another  system  for  knowledge  acquisition  in  medicine  which  lets 
experts  build  knowledge  bases  without  a  knowledge  engineer  to  act  as  intermediary.  The 
QMR  knowledge  base  is  one  of  the  most  comprehensive  knowledge  bases  in  use  for 
medical  decision  support  for  diagnosis  of  internal  medicine  diseases.  Knowledge  base 
information  consists  of  textual  descriptions  of  a  disease  called  disease  unit.  This 
representation  is  different  than  the  visual  approach  of  APPRENTICE,  and  of  course  the 
QMR  system  is  not  a  planning  system. 

There  are  other  knowledge  acquisition  systems  that  take  a  different  approach  than 
APPRENTICE.  MORE  [Kahn  85]  is  a  system  which  automatically  interviews  experts  for 
the  information  they  use  to  solve  diagnostic  problem.  ASK  [Gruber  89]  is  a  system  that 
elicits  justifications  from  experts  and  generates  strategy  rules.  It  is  used  with  the  system 
MUM,  which  is  a  knowledge  system  that  concentrates  on  the  strategic  aspects  of  diagnosis 
of  chest  pain.  DACRON  [Mahling  89]  is  a  visual  knowledge  acquisition  tool  for  the 
POLYMER  planning  system.  SALT  [Marcus  86]  is  a  system  that  assists  in  knowledge 
acquisition  for  incremental  design  tasks  such  as  configuration.  The  system  uses  a  propose- 
and-revise  problem  solving  strategy.  ROGET  [Bennet  85]  is  a  program  to  assist 
knowledge  engineers  and  domain  experts  in  developing  EMYCIN-based  expert  systems. 
KNACK  [Klinker  88]  is  a  knowledge  acquisition  tool  for  building  knowledge  bases  to 
generate  reports.  This  system  is  interesting  because  it  is  sample-based:  the  expert  provides 
a  sample  repot,  and  the  system  analyzes  and  generalizes  the  sample. 
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To  help  explain  some  differences  between  tire  above  systems,  six  system  characteristics  are 
outlined  below.  These  characteristics  are  then  shown  in  a  chart  for  comparison. 


Feedback  Type  -  type  of  feedback  the  system  gives  (textual,  animated,  graphs). 

Knowledge  Developed  -  type  of  knowledge  the  system  can  acquire  (static  and/or  dynamic 
knowledge). 

Domain  Type  -  types  of  domains  the  system  is  used  for. 

Generate-Test  Cycle  •  development  done  in  generate  and  test  mode  in  conjunction  with  the 
target  expert  system. 

Mode  Type  -  system-directed  acquisition  or  user-directed  acquisition. 

Knowledge  Acquired  -  acquire  incremental,  factual  and/or  better  performance  knowledge. 
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Figure  2.1:  Table  of  differences  among  knowledge  acquisition  systems 


Finally,  knowledge  base  development  environments  such  as  KREME  [Abrett  87], 
ONTOS  [Nirenburg  88],  and  CYC  [Lcnat  86]  elicit  mostly  static  or  factual  knowledge. 
These  systems  investigate  how  to  organize  and  maintain  large  knowledge  based  systems. 
Handling  large  knowledge  bases  is  a  future  direction  of  the  APPRENTICE  system. 
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The  major  difference  between  APPRENTICE  and  other  systems  is  that  APPRENTICE 
enables  users  to  model  the  physical  world  with  their  own  defined  representations.  Also,  the 
APPRENTICE  system  is  interactive  with  the  Prodigy  planning  system  providing  a  closed 
loop  for  development  and  utilization  of  a  knowledge  base.  Although  the  planner  and 
acquisition  tool  are  integrated  the  APPRENTICE  techniques  are  also  described 
independent  of  a  planning  system. 
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Chapter  3  -  Apprentice  Architecture 


The  APPRENTICE  system  can  be  subdivided  into  three  components:  the  graphical 
interface  component,  Framegraphics;  the  domain  building  component.  Domain  Builder; 
and  the  planning  system.  Prodigy.  I  built  the  Framegraphics  system  and  the  Domain 
Builder  far  this  research.  Figure  3.1  depicts  the  relationships  among  the  components  in 
mote  detail.  In  this  chapter,  I  will  discuss  in  some  detail  the  various  components  of  the 
system. 
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Figure  3.1:  Diagram  of  the  APPRENTICE  system. 


3.1  Prodigy 

Prodigy  [Minton  89]  is  a  domain- independent  problem-solver  system  used  primarily  as  a 
testbed  for  research  in  planning,  machine  learning,  and  knowledge  acquisition.  It  uses  basic 
means-ends  analysis  in  planning  for  high-level,  symbolic  domains.  Since  its  inception,  the 
Prodigy  system  has  continued  to  develop.  The  first  version  of  the  system.  Prodigy  2.0, 
was  a  linear  planner  (i.e.,  it  did  not  allow  interleaving  of  goals)  [Minton  89].  The  next 


version  of  Prodigy,  Nolimit,  was  a  non-linear  planner  which  provided  interleaving  goals 
[Veloso  92 J.  The  syntax  of  the  two  systems  is  slightly  different,  with  the  biggest  difference 
being  that  Nolimit  requires  all  objects  used  in  a  domain  to  be  explicitly  defined  in  the 
syntax  (see  section  3.1 2). 

The  basic  design  of  APPRENTICE  was  able  to  accommodate  both  planners. 
APPRENTICE  was  first  developed  using  Prodigy  2.0  but  now  works  with  Nolimit. 
Porting  APPRENTICE  to  the  different  problem  solvers  required  very  little  modification. 

The  basic  functioning  of  APPRENTICE  with  a  problem  solver  will  be  discussed  later. 
This  section  will  outline  die  Nolimit  Prodigy  planner.  Then  a  simple  trace  of  die  planner 
solving  a  problem  will  be  described;  and  finally,  an  outline  of  the  input  language  will  be 
presented.  This  is  die  planner  which  was  used  in  the  studies  described  later. 

3JJ  Operation  of  the  Prodigy  Planner 

A  rfnmain  theory  in  Prodigy  consists  of  operators,  inference  rules,  and  search  control  rules. 
These  rules  are  essentially  If-Then  rules.  The  left-hand  sides  of  these  rules  arc  represented 
by  a  Prodigy  Description  Language  (PDL).  This  language  is  a  form  of  first-order  predicate 
logic.  PDL  allows  conjunctions,  disjunctions,  negations,  and  existential  and  universal 
quantifications.  APPRENTICE  automatically  generates  conjunctions  and  existential 
quantification  from  its  pictorial  representation.  A  disjunction  can  be  modelled  as  multiple 
rules,  and  negations  can  be  represented  explicitly.  Universal  quantifications  are  not 
automatically  generated  by  the  APPRENTICE  system  but  can  be  added  manually  to  the 
generated  code.  Thus  far,  none  of  the  domains  that  were  built  in  APPRENTICE  needed 
universal  quantification.  Therefore  a  lot  of  resources  were  not  devoted  to  developing  a 
simple  way  to  generate  and  depict  universal  quantification  graphically. 

Operators  and  inference  rules  are  factual  knowledge  that  modifies  the  state  as  the  planner  is 
solving  a  problem.  Although  inference  rules  do  not  correspond  to  external  actions,  they  can 
be  mndeMfid  a s  operators.  On  the  other  hand,  search  control  rules  are  control  knowledge 
that  directs  the  search  for  the  solution.  This  thesis  deals  mainly  with  the  creation  of  factual 
knowledge.  Developing  search  control  knowledge  is  discussed  in  section  6.3. 

Operators  have  three  parts:  the  parameter  list,  the  preconditions,  and  the  effects.  The  effects 
part  of  an  operator  either  or  deletes  a  predicate  from  the  state.  When  given  an  initial 
and  goal  state  along  with  a  domain  definition,  the  Prodigy  planner  tries  to  determine  a 
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sequence  of  actions  (instantiated  operators)  that  when  applied  in  a  given  order  will  change 
the  initial  state  to  achieve  the  goal  state.  The  search  tree  initially  begins  with  a  single  node 
representing  a  set  of  state  predicates  in  the  initial  state  and  a  set  of  goal  predicates  to  be 
achieved.  To  move  from  the  initial  state  to  the  goal  state,  die  tree  is  expanded  by  repeating 
two  phases:  the  decision  phase  and  the  expansion  phase. 

The  first  step  during  the  decision  phase  is  to  choose  a  node.  The  second  step  is  to  choose  a 
goal  to  focus  on,  which  becomes  the  current  goal.  The  third  step  is  to  choose  an  operator  to 
achieve  die  current  goal.  The  final  step  is  to  choose  bindings  for  the  variables  in  the 
operator  (this  is  called  instantiating  the  operator).  The  default  search  strategy  for  this 
decision  process  is  depth-first  search  but  can  be  guided  by  search  control  rules.  These 
search  control  rules  are  used  to  prune  the  search  tree  [Minton  89]. 


Figure  3.2:  Functional  view  of  Prodigy. 


During  the  expansion  phase  a  new  node  is  created.  If  the  instantiated  operator's 
preconditions  are  satisfied  by  the  state,  that  the  operator  is  applied.  Otherwise,  the  system 


subgoals  (changing  the  current  goal)  on  an  unmatched  precondition:  adding  the  old  current 
goal  to  the  set  of  goals  that  are  pending;  or  subgoals  on  a  previously  unachieved  goaL 

When  a  state  is  achieved  in  which  all  of  the  top-level  goals  are  satisfied,  the  problem  is 
solved.  Figure  32  shows  a  high  level  flowchart  of  the  Prodigy  system. 

3J.2  Prodigy  Format 

Writing  a  domain  in  Prodigy  requires  several  steps.  The  user  most  create  object  prototype 
definitions,  predicates  representing  relations  among  objects,  operators,  instances  of  objects, 
start  states,  and  goal  states.  An  example  of  a  domain  being  built  will  be  shown  using  a 
slight  variation  of  die  blocks  world  domain.  The  blocks  world  domain  consists  of  a  set  of 
blocks  that  can  be  stacked  on/unstacked  off  each  other  or  put-down  on/picked-up  off  the 
table.  In  this  Mocks  world  the  table  and  the  arm  are  represented  explicitly. 

3J2J  Defudnr  Object  Types 

Object  definitions  for  a  domain  are  defined  with  the  "is-a"  function.  These  definition  types 
tell  Prodigy  the  name  of  the  objects  in  die  domain.  In  the  example  blocks  world,  the  user 
will  need  block  objects,  table  objects,  and  arm  objects.  An  object  definition  has  the 
following  format:  <  is-a  abjmct~naam  type)  .  In  this  example,  these  objects  are  defined 
as  follows: 

(IS-A  Block -aodal  TOPS) 

(IS-A  Tabla-aodal  TO? 8) 

(IS-A  Ara-nodal  TOPS) 

Figure  3-3:  IS-A  Definitions  for  die  blocks  world 

3JJJ  HeSmat  EndkatesmiOotrotoa 

The  operator  defines  legal  transitions  between  states.  Each  operator  has  a  precondition  that 
must  be  satisfied  before  the  operator  can  be  applied  and  an  effects- list  that  describes  how 
the  application  of  the  operator  changes  the  state.  Specifically,  the  effects-list  indicates  the 
atomic  formulas  that  are  added  to  and/or  deleted  from  the  state  when  the  operator  is 
applied.  These  preconditions  and  effects  are  represented  by  predicates.  Predicates  represent 
relationships  between  objects.  In  order  to  allow  an  operator  to  work  in  a  generic  way,  the 
predicates  use  variables.  These  variables  match  different  objects  in  the  state  definition,  "o" 
surrounding  a  name  represents  a  variable.  An  operator  format  is  shown  below: 
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(OPERATOR  nana-of-oparaeor 
(IIMIB 

( (<varl>  cypal) 

(<vax2>  cypal) 

(<var3>  typa3)  . . .) 

(PREC0MD6 

(AMD  (pradl  <varl>  <vmr2>) 

(prad2  <var2>  <var3>) ) . . ) 

isrrwcTs 

( (DSL  (pradl  <varl>  <var2>) ) 

(ADD  (prad3  <vmrl>  <var3>) ) )  ...  ) 

Figure  3.4:  Operator  format. 


Operators  in  die  blocks  world  are  defined  in  Figure  3.5.  Consider  the  operator  Pick-up.  In 
its  P ARAMS  list  the  variable  <ob>  is  of  type  block-model,  <tabi«>  is  of  type  table- 
model,  and  <axm>  is  of  type  arm-model  In  the  precondition,  die  predicate  (clear  <ob>) 
checks  that  the  block  <ob>  has  nothing  on  top  of  it;  the  predicate  (on-table  <tabie> 
<ob>)  checks  to  see  if  the  variable  <ob>  is  on  the  <table>;  and  the  predicate  (empty 
<arm>)  checks  that  the  <arm>  is  empty.  Finally,  if  all  of  these  predicates  are  true  for  a 
particular  block,  tabic  and  arm  in  the  current  state,  then  the  matching  block  is  picked  up  off 
the  table  by  the  arm.  Also  the  predicates  in  the  effect  list  update  the  state  by  dclcting/adding 
the  relevant  predicates. 


(OPERATOR  PICK-UP 

(partw  ( (<ob>  BLOCK-MODEL) 

«cmbl«>  TABLE -MODEL) 

(<am>  ARM-MODEL ) ) ) 

(praconda  (and  (claar  <ob>) 

(on-tabla  <tabla>  <ob>) 
(anpty  <«*>!)) 

(affect*  ((dal  (on-cable  <cable>  <ob>) ) 

(dal  (claar  <ob>) ) 

(dal  (anpty  <arn>) ) 

(add  (holding  <arm>  <ob>) ) ) ) ) 

(OPERATOR  UNSTACK 

(parson  ( (<ot»  BLOCK-MODEL) 

(<undarob>  BLOCK-MODEL) 

(<arm>  ARM-MODEL) ) ) ) 

(praconda  (and  (on  <ob>  <undarob>) 

(claar  <ob>) 

(aapty  <an*>) ) ) 

(affacca  ( (dal  (on  <ob>  <undarob>) ) 

(dal  (claar  <ob»> 

(add  (holding  <arn>  <ob>) ) 

(add  (claar  <undarob>) ) ) ) ) 

(OPERATOR  POT-DOWI 

(Parana  ( (<ob>  BLOCK-MODEL) ) 

(<U»  ARM-MODEL) 

(<cabla>  TABLE-MODEL) ) ) 

(praconda  (bolding  <arn>  <ob>) ) 

(af facta  ((dal  (holding  <arn>  <ob>) ) 

(add  (claar  <ob>) ) 

(add  (wpty  «an>)l 

(add  (on-tabla  <tabla>  <ob>) ) ) ) ) 

(OPERATOR  STACK 

(Parana  ( (<ob>  BLOCK- MODEL) 

(<undarob>  BLOCK-MODEL) 
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(<»rm>  ARH-MOOXL)  ) ) ) 

(pracoods  (and  (clear  <undarob>) 

(holding  <ara»  <ob>))) 

(a££a eta  ( (dal  (holding  <irm>  <ob» ) 

(dal  (clear  <underob>) ) 

(add  (dear  <ob>) ) 

(add  (on  <ob>  <undarob>) ) ) ) ) 

Figure  33:  Operators  for  the  blocks  world. 

XlldJkfminr  a 

After  the  domain  is  developed,  the  problems  that  the  user  wants  to  solve  must  be  defined. 
A  particular  problem  is  defined  by  instances  of  objects,  a  start  state,  and  a  goal  expression, 
which  is  a  subset  of  the  entire  goal  state.  Below  is  a  problem  in  the  blocks  world.  The 
instances  are  A,  B,  C,  A_Table,  and  An_Arm.  The  goal  is  to  get  A  on  A_Table,  Bon  A, 
and  C  on  B.  The  start  state  has  Blocks  A,  B,  and  C  on  A_Table  and  An.Arm  empty. 

(HAS-INSTANCES  Block -nodal  A) 

(HAS -INSTANCES  Block-nodal  B) 

(HAS- INSTANCES  Block-nodal  C) 

(HAS -INSTANCES  Table-model  AJTable) 

(HAS- INSTANCES  Aza-nodal  An_Arm) 

(GOAL  (on  B  A) 

(On  C  B) 

(On-cabla  A) ) 

(STATE  (AND  (On -Table  A_Tabla  A) 

(On -Table  AJTable  B) 

(On-Tabla  A-Tabla  C) 

(Claar  A) 

(Claar  B) 

(Claar  C) 

(Brpcy  An_Am)))) 

Figure  3-6:  Problem  Definition  for  the  blocks  world. 


3.13  Example  Trace 

Using  the  domain  and  problem  definition  from  the  previous  section.  Figure  3.7  depicts  a 
partial  solution  trace  of  die  Prodigy  planner.  The  planner  uses  means-ends  analysis  to  solve 
the  problem.  The  planner  first  determines  which  operators  achieve  the  final  goal.  If  the 
operators  are  not  applicable,  it  then  determines  which  operators  establish  L:e  pre-state  of  the 
previously  related  operator.  This  continues  recursively  until  a  sequence  of  operators  is 
found  that  is  applicable  to  the  initial  state  and  achieves  the  goal. 


The  trace  shows  a  segment  of  the  solution  path  in  which  the  system  backtracks.  Each  line 
contains  information  about  the  step  in  the  means-ends  analysis  being  fried.  The  To# 
displays  the  node  name  of  the  solver’s  current  position.  The  next  part  of  the  line  is  either  a 
goal  to  be  worked  on,  an  instantiated  operator  to  try  and  satisfy,  or  an  operator  to  apply. 
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The  applied  operators  are  capitalized.  If  there  are  additional  choices  available  at  a  particular 
node  that  information  is  displayed  in  the  form  of  a  list  (e.g.,  goal-choices-left  or  ops- left). 

The  first  part  of  the  partial  trace,  starting  with  the  *****,  does  not  succeed  because  it  cannot 
satisfy  the  goal  (on  b  a)  with  c  already  stacked  on  b.  This  causes  a  condition  where  a  state 
would  be  repeated  and  the  solution  search  would  end  up  in  an  infinite  loop,  therefore  the 
system  backtracks  and  reorders  the  top  level  goals.  With  this  reordering,  the  second  part  of 
the  trace  leads  to  a  solution.  The  solution  found  is  listed  at  the  end  of  the  trace. 

After  several  other  taamptsd  solutions  during  problem  solving 


tnl  (done) 
cn2  (‘finish*) 

I  tn3  (on  c  b) 

I  goal-choices-left:  ((on  b  a)) 

I  tn4  (stack  b  c  an_arn) 

I  I  cnS  (holding  c  an_an) 

I  I  tn6  (pickup  an_an  c  a_table) 

I  I  ops-left:  ( (unstack  b  c  an_an)  (unstack  c  c  an_arm)  (unstack  a  c 

an.ana)  ) 

I  I  TN7  (PICKUP  ANJUUi  C  A_TABLE) 

I  TN8  (STACK  B  C  AN^ARM) 

I  tn9  (on  b  a) 

I  tnlO  (stack  a  b  an_an) 

I  tnll  (holding  b  an_an) 

I  tn48  (unstack  a  b  an_ann) 

**•  STATE  LOOP  “* 

**“••**  BACKTRACK  TO  NODE  T2  *****••*•*• 

tn2  (‘finish*) 

I  tn56  (on  b  a) 

I  tn57  (stack  a  b  an_arm) 

I  I  tn58  (holding  b  an_an) 

I  I  tnS9  (pickup  an_an  b  a_table) 

I  I  ops-left:  ((unstack  b  b  an_an)  (unstack  c  b  an_arm)  (unstack  a  b 

an_arm) ) 

I  I  TN60  (PICKUP  ANJVRM  B  A_TABLE ) 

I  TN61  (STACK  A  B  AN_ARM) 

I  tn62  (on  c  b) 

I  tn63  (atack  b  c  an_an) 

I  I  tn64  (holding  c  an_an) 

I  I  tn65  (pickup  an_an  c  a_table) 

I  I  ops-left:  ((unstack  b  c  an_arm)  (unstack  c  c  an.arm)  (unstack  a  c 

an.aral  ) 

I  I  TN66  (PICKUP  AN_ARM  C  A.TABLE) 

I  TN67  (STACK  B  C  AN_ARM) 

TN68  ( •FINISH*) 


This  is  the  solution  found: 

(pickup  arm  b  table) 

(atack  a  b  an) 

(pickup  an  c  table) 

(atack  b  c  an) 

(•finish*) 

Figure  3.7:  Partial  solution  trace  in  Prodigy  for  the  blocks  example  domain. 
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3.2  Domain  Builder 

The  domain  builder  allows  an  expert  to  graphically  represent  and  create  a  domain.  This 
graphical  process  resembles  an  expert's  experience  more  closely  than  writing  a  domain  in 
textual  form  as  described  in  the  previous  section.  Once  a  graphical  domain  is  defined,  the 
domain  builder  then  automatically  translates  this  graphical  representation  into  PDL,  which 
is  loaded  and  run  in  the  Prodigy  planner.  The  graphical  system  is  tightly  integrated  with  the 
planner,  providing  intuitive  domain  development  This  section  outlines  the  implementation 
of  graphical  knowledge  acquisition,  then  discusses  the  primitives  that  are  needed  for  the 
process  independent  of  the  actor?  implementation. 

32.1  High  Level  Window  Description 

In  developing  a  domain  using  the  domain  builder,  several  things  have  to  be  defined: 
objects,  relationships  between  the  objects,  operators  to  change  sets  of  relationships,  and 
initial  and  goal  stales  for  defining  problems.  In  die  domain  builder,  separate  windows  are 
used  to  develop  each  particular  element.  The  windows  are: 

Model  Window  -  Allows  objects  in  a  domain  to  be  defined. 

Relation  Window  -  Allows  relationships  between  objects  to  be  defined. 

Operator  Window  -  Allows  state  changes  to  be  defined. 

State  Window  -  Allows  the  definition  of  a  start  state  and  a  goal  state. 

Problem  Window  -  Allows  the  setup  and  running  of  a  problem  to  be 
solved. 

Apprentice  Window  -  Allows  the  manipulation  of  a  domain. 

The  four  subcomponent  windows  (object,  relation,  operator,  and  state)  are  used  to  define  a 
domain.  The  Problem  Window  is  used  to  create  a  problem  that  will  be  run  in  the  Prodigy 
planner.  The  Apprentice  Window  is  used  for  domain  manipulation:  saving,  deleting,  and 
loading  domains. 

All  the  windows  have  a  similar  MacDraw™[MacDraw  89]  style  interface,  which  is 
mouse-  and  object-oriented.  This  similarity  allows  the  user  to  interact  with  each  window  in 
a  consistent  manner.  A  single  mouse  "click'1  selects  an  object;  "click  and  drag"  moves  an 
object;  and  "double-click"  allows  an  object's  name  to  be  edited  or  a  domain  element  to  be 
made  available  for  editing.  Windows  are  reconfigurable.  All  objects  in  the  window  can  be 
easily  moved,  and  their  locations  saved,  so  that  each  window  can  be  customized  to  a  user’s 
preference. 
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F»ch  window  has  a  work  area  in  which  a  particular  domain  element  type  (objects, 
relations,  operators,  and  states)  can  be  developed.  This  work  area  allows  elements  to  be 
developed  one  at  a  time.  For  example,  in  Figure  3.8  the  Model  Window’s  work  area  is 
used  for  building  a  block  object  To  build  or  modify  a  different  object  the  block  object  is 
iconified  and  another  object  can  be  manipulated.  This  iconifying  process  takes  the  sub-parts 
of  the  current  element  from  the  work  area  and  represents  all  of  the  information  in  a  single 
name  that  can  be  placed  anywhere  on  the  screen.  An  element  can  be  edited  again  by  double 
eliciting  on  its  iconified  name.  This  action  places  die  original  element  in  the  window,  and 
the  sub-parts  of  die  double-clicked  element  are  put  into  the  work  area  ready  to  be  modified. 

33J.1  Model  Window 

The  Model  Window  is  used  to  create  prototype  objects  of  the  domain  along  with 
connection  points  (i.e.,  location  in  which  the  objects  will  interact  with  one  another).  These 
objects  are  then  used  in  defining  relations,  operators,  and  states.  The  work  area  allows  one 
object  at  a  time  to  be  developed.  The  work  area  has  a  very  simple  object-oriented  drawing 
package  type  interface,  fv  this  work  area  lines,  text,  instance  names,  and  connection  points 
can  be  added,  detet^i,  or  edited  for  an  object  Each  element  in  the  work  area  can  be 
nruinipnlatBri-  To  create  a  particular  element  type,  the  user  selects  the  corresponding  box 
from  the  palette  on  the  left  of  the  work  area.  In  Figure  3.8,  the  box  element  has  been 
selected. 

Once  all  the  modifications  to  an  object  are  made,  then  the  results  can  be  saved,  thus 
incorporating  the  changes  into  the  prototype  object  Objects  can  also  have  attribute 
information  attached  to  them.  Attributes  are  added  to  objects  by  double  clicking  on  the 
Attn  field.  This  brings  up  a  popup  editor  which  allows  attribute  information  to  be  added. 
This  attribute  information  is  specified  by  an  attribute  name  and  the  variables  with  type 
definitions  that  follow.  In  Figure  3.8  the  attribute  of  the  block  is  weight,  and  its  value  is 
some  number.  During  the  generation  of  a  state  or  operator  this  attribute  will  translate  into 
(weight  <biocJcxx>  <ibs>) .  Each  individual  object  can  be  edited  to  include  or  exclude 
attributes. 
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MODEL-WINDOW 


Ml 

PW-HIS 

EP 

Figure  3  A  Model  Window:  editing  the  block-model  object 


12 .1  7  SdOUIL  Window 

The  Relation  Window  allows  the  user  to  build  relations  between  objects  and  automatically 
generates  a  predicate  from  the  objects.  These  relations  are  used  in  the  automatic  code 
generation  during  die  operator  and  state  definition.  This  allows  users  to  define  much  of  the 
rule  and  state  knowledge  graphically,  and  the  system  is  able  to  check  and  flag 
inconsistencies  in  the  definitions.  Figure  3.9  is  an  example  of  the  Relation  Window. 


In  the  Relation  Window,  object  prototypes  that  the  system  knows  about  are  pictured 
around  the  outside  of  die  work  area.  These  objects  can  be  moved  by  the  user  to  any 
position  outside  the  work  area.  Relations  are  defined  by  dragging  prototype  objects,  one  at 
a  time,  into  the  work  area.  This  produces  instances  of  the  objects.  Instances  are  then 
connected  together  by  their  connection  points  to  represent  the  relation.  This  visual  picture 
translates  into  a  textual  list  of  the  relation  name  and  the  objects  in  the  relation.  This  textual 
representation  is  a  predicate  in  PDL.  These  predicates  are  used  in  operator  and  state 


Figure  3.9  represents  the  relation  on-Tabi«  between  a  block  and  a  table.  The  block  and 
table  are  connected  together  at  their  connection  points.  This  connection  is  d*signat«ri  by  a 
line  joining  them.  During  the  state  and  operator  definitions,  whatever  a  block  and  a  table 


Page  26 


are  connected  together  in  this  fashion,  the  on-Tabia  predicate  will  be  automatically 
generated. 

To  the  left  of  the  work  area  are  the  object  prototypes  that  were  created  in  the  Model 
Window.  At  the  top  of  the  window  are  two  relations  that  have  already  been  defined:  on  and 
Holding.  The  machine-generated  PDL  predicate  (on-table  <block876>  <tabie564>) 
is  in  die  window  and  can  be  directly  edited  by  the  user.  The  Dep-List  in  the  window  allows 
the  user  to  define  a  dependency  order  among  the  objects  that  is  used  for  the  solution 
animation  The  Dep-List  will  be  discussed  in  section  3.2 2. 


Ochar  Relations 


Figure  319;  Relation  Window:  developing  the  On-Table  relation. 


The  Relation  Window  also  allows  a  user  to  define  a  negated  connection  existing  between 
two  objects  (U.,  arm  and  block).  A  connection  is  negated  by  selecting  the  connection  point 
to  be  negated  then  clicking  the  Not  Connected  button.  In  Figure  3.10,  when  an  arm  is  not 
holding  a  block,  the  empty  relation  holds.  The  square  around  the  arm  connection  point 
signifies  that  the  connection  has  been  negated. 
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Figure  3JUk  Relation  Window:  showing  a  negated  connection. 


RELATION-WINDOW 


|~~)  Delete  Inst  Q  Not  Connected  On  Oa-Table 

Holding  Empty 


DONE  RELATION:  Clear 


Peg-List 


71 

<ahta>  1/ 


(Gear  <block876>) 


Figure  3.11:  Relation  Window:  showing  a  non-connection  definition. 


Finally,  there  is  a  way  to  describe  a  relation  when  an  object  is  not  connected  to  any  other 
object  This  is  done  by  selecting  a  connection  point  on  an  object  then  clicking  the  Not 
Connected  button.  This  signifies  that  when  nothing  is  connected  to  that  connection  point 
the  relation  holds.  In  Figure  3.11  the  clear  relation  is  shown.  This  relation  is  applicable 
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when  there  is  nothing  connected  to  the  top  of  a  block  (i.e.,  no  block  is  on  top  of  the  block, 
and  the  block  is  not  being  held  by  an  aim). 

32  JJ  Operator  Window 

The  Operator  Window  allows  the  graphical  creation  of  operators  or  rules  in  the  domain  and 
automatically  generates  Prodigy  operator  code.  In  this  window  the  work  area  is  divided 
into  two  regions:  before  and  after.  When  all  the  relations  in  the  before  region  are  true  for  a 
particular  state,  then  the  operator  transforms  the  matched  objects  into  the  relations  in  the 
after  region.  Therefore,  the  before  region  defines  the  conditions  that  have  to  be  true  before 
the  operator  is  applied,  and  the  after  region  imolicitly  defines  the  transition  between  states 
when  die  operator  is  applied. 

An  operator  is  defined  by  building  a  picture  of  its  before  and  after  states.  This  creation 
process  is  similar  to  defining  a  relation.  To  create  an  operator,  the  relevant  prototype  objects 
are  dragged  into  the  before  region  of  the  work  area  and  connected  together  to  represent  the 
before  state.  The  user  can  then  copy  these  objects  to  the  after  region  by  clicking  the  Copy 
Pre  State  button.  This  will  also  link  the  similar  objects  together.  Thus,  if  the  name  of  an 
object  in  the  before  state  is  changed,  then  the  corresponding  object's  name  in  the  after  state 
is  also  changed,  and  vice  versa.  Objects  can  also  be  added  directly  to  a  region  with  the  drag 
technique,  but  these  objects  are  not  linked  with  any  other  objects.  To  link  two  similar 
objects  in  the  before  and  after  region,  the  user  should  select  die  desired  objects  and  click  the 
Create  Link  button.  Once  the  before  and  after  states  are  defined,  the  system  can 
automatically  compute  the  preconditions  and  the  effects  for  the  operator  in  PDL.  This 
computation  is  displayed  so  that  the  user  can  make  modifications  if  needed.  User 
modifications  made  to  the  generated  code  are  recorded  and  used  when  the  system 
automatically  regenerates  the  operator  due  to  graphical  modifications  the  user  makes. 

In  Figure  3.12  the  operator  Pickup  is  being  defined.  Similar  to  the  relation  window,  the 
prototype  objects  are  positioned  around  the  work  area.  Note  that  names  of  objects  in  the 
operator  have  o  around  them.  This  represents  a  variable  name.  Figure  3.13  shows  the 
generated  code  for  the  Pickup  operator. 
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OPERATOR-WINDOW 


Figure  3.12:  Pickup  operator  before  code  is  generated. 


OPERATOR-WINDOW 


I  I  Ddalalwt  Q  Copy  Pre  State  Q  Create  fink 


Name:  Pickup  (dd(ooHaUo<bicdc><mMe>)) 

(ckv<tSock>)  DONE  (del  (empty  <wm>)) 

(empty  <ann>)  I  (del  (deer  <bioek>)) 

(add  (hotting  <ann>  -chkxdo)) 


7 

<tafaie>  \s 


= liii  iiM»  §  §4  # 


Figure  3JL3:  Operator  Window:  showing  the  Pickup  operator  with  automatically  generated 

Prodigy  code. 


The  State  Window  allows  the  graphical  description  of  a  state  and  automatically  generates 
the  Prodigy  state  code  to  reflect  the  depiction.  The  initial  state  is  a  description  of  the  state  at 
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the  beginning  of  planning.  The  definition  procedure  is  similar  to  the  other  process.  The 
state  is  defined  by  instances  of  objects  and  their  connections.  The  user  drags  prototype 
objects  into  the  work  area  and  connects  them  together.  From  this  visual  representation  the 
system  generates  an  initial  state  code  definition  automatically.  Figure  3.14  is  an  initial  state 
that  has  three  blocks  on  a  table.  The  automatically  generated  state  code  is  also  part  of  the 
State  Window. 


STATE- WINDOW 


Ddetolnat  Q]  Copy  Stale 


/ 

X  ' 

/- 

X 

MBiillfe 

DONE  Name:  Initial-State-1 

Pp&UB. 

/  *  \ 


Slate 

(oo-tabie  A  A_Tabte) 
(ao-table  B  A_Table) 
(oo-tabie  C  A_Tabie) 
(dear  A) 

(clear  B) 

(clear Q 
(empty  An_Ann} 


Figure  3.14:  State  Window:  defining  Initial-State- 1. 


The  goal  state  is  a  little  different  from  the  initial  state  because  the  goal  state  can  have  a 
subset  of  the  objects  in  the  initial  state.  Figure  3.15  shows  a  goal  state,  again  with  the 
automatically  generated  state  description.  The  state  definition  can  also  be  directly  edited  by 
die  user.  Again,  changes  made  to  this  definition  are  remembered  and  are  applied  when  the 
system  automatically  regenerates  the  state  description  due  to  graphical  changes. 
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STATE*  WINDOW 


Q  DtfotoiMt  0  Copr State  imtial-State-1 


Name:  Goal-State- 1 


/  *  \ 
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(on  BA) 
(oaCB) 
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Figure  3.15:  State  Window:  defining  Goal-State- 1. 


In  the  Problem  Window  tin  user  defines  the  problem  that  is  to  be  solved  and  sees  the 
animation  as  die  system  plans  the  solution.  In  order  to  solve  a  problem.  Prodigy  needs  to 
be  given:  1)  the  operators  that  are  to  be  used,  2)  the  initial  state  for  the  problem  and  3)  the 
goal  state  for  the  problem.  These  are  specified  separately,  providing  flexible  problem 
specification.  For  example,  one  may  specify  several  problems  with  the  same  initial  state 
but  different  goal  states,  or  test  only  a  subset  of  the  available  operators.  Figure  3.16  shows 
the  Problem  Window  based  on  the  information  we  previously  defined.  By  selecting  the 
Execute  Problem  button,  the  current  problem's  code  is  loaded  into  Prodigy  and  executed. 
As  die  planner  solves  the  problem,  the  solution  is  graphically  animated,  as  shown  in  the 
next  section. 
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Figure  3.16:  Problem  Window:  displaying  the  initial  and  goal  state  definitions. 


3JJ.6  Exmak.  Trace 

Figure  3.17  is  an  example  trace  of  the  system  solving  the  problem  that  we  previously 
defined.  Hie  domain  and  problem  definition  are  the  same  as  the  textual  domain  that  we 
defined  previously.  The  partial  trace  below  shows  die  steps  the  system  takes  in  order  to 
achieve  the  goal  state.  The  system  starts  out  by  stacking  the  blocks  in  the  wrong  order. 
Later  on,  the  system  backtracks  and  reorders  die  goals.  The  system  then  stacks  the  blocks 
in  the  correct  order  and  solves  the  problem. 

The  solution  for  the  defined  problem  is  to: 

(pickup  arm  b  cabla) 

(stack  a  b  arm) 

(pickup  arm  c  tabla) 

(stack  b  e  arm) 

Another  debugging  tool  allows  the  user  to  apply  instantiated  operators  one  by  one.  When 
applying  an  operator,  if  preconditions  are  not  satisfied  in  the  particular  state,  a  message  is 
displayed  containing  die  relations  that  were  not  matched  in  the  precondition.  This  stepping 
process  helps  the  user  verify  that  an  operator  is  performing  as  believed.  Of  course,  all 
graphical  animation  is  also  updated  when  an  operator  is  applied  successfully. 
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Figure  3.17:  Visual  animation  showing  blocks  world  problem  being  solved.  These  are 
several  snapshot  views  of  state  changes  with  applied  operator  or  backtrack  command. 


322  Animation  Algorithm 

The  animation  algorithm  provides  the  ability  to  visually  trace  die  progress  of  a  solution 
during  planning.  This  is  done  systematically,  as  shown  by  Figure  3.18.  Once  a  problem  is 
ready  to  be  solved,  then  the  animation  routine  begins.  First  the  graphical  object  information 
is  initialised;  die  graphical  representation  of  each  object  is  displayed  in  the  problem  window 
at  a  location  based  upon  the  initial  state  definition.  A  list  of  all  the  graphical  objects  with 
supporting  information  is  compiled  and  stored  for  animation.  The  graphical  dependencies 
and  relative  position  for  each  object  in  all  the  relations  are  computed  and  stored.  In  the  next 
step,  die  PDL  definition  for  the  domain  is  collected  and  prepared  to  be  loaded  into  the 
Pnxfigy  planner.  Once  the  initialization  is  finished,  die  PDL  code  is  loaded  into  the  planner 
and  planning  begins.  During  the  planning  phase,  the  planner  will  update  the  graphical 
display  as  operators  are  applied.  This  update  procedure  will  be  explained  later  in  more 
detail.  If  the  planner  backtracks,  then  the  screen  flashes  and  die  new  state  being  worked  on 
is  drawn. 
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Figure  3.18:  Animation  diagram. 


The  key  to  this  trace  animation  is  getting  the  information  about  die  object's  interaction  from 
die  relation  information.  When  an  operator  adds  or  deletes  a  set  of  relations  in  the  current 
state,  the  relation  information  is  used  to  update  die  graphical  display.  Therefore,  the  initial 
information  needed  from  each  relation  is:  1)  what  are  the  object  types  in  the  relation;  2) 
what  are  the  dependencies  between  objects;  and  3)  what  are  the  relative  distances  between 
the  objects.  A  short  example  illustrating  the  information  needed  and  how  it  is  used  follows. 

♦ 

The  objects  in  a  relation  are  ordered  according  to  the  object's  dependency  with  the  other 
objects.  This  order  is  user-defined  and  represented  in  the  Dep-list  of  the  relation  window. 
Figure  3.19  shows  die  relation  window  with  the  definition  of  the  "on”  relation.  The  bottom 
object  in  the  Dep-List  is  the  most  stable  object  during  movement  Thus  in  Figure  3.20  the 
top  block  is  moved  to  a  location  relative  to  the  bottom  block.  The  graphical  distance 
between  the  two  objects  is  used  when  the  objects  are  animated  in  the  solution  trace. 
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Figure  3.19:  The  Relation  Window  with  the  on  relation 


Figure  3.20  shows  three  snapshots  of  an  animation  trace.  The  first  snapshot  shows  B  tight 
before  an  operator  is  applied  to  add  (on  b  a}.  Snapshot  two  shows  the  result  of  that 
relation  being  added  to  the  state.  See  how  block  B  moved  relative  to  the  position  of  block 
A.  Further  in  the  solution  trace,  snapshot  three  shows  the  after  effects  of  adding  the  (on  c 
B)  relation  to  the  state.  In  tins  case  MockC  moved  relative  to  the  position  of  block  B. 


Bafora  After  La.fr,  After 

(on  BA)  (on  B  A)  (on  C  B) 


Ftgnre  3JXk  Adding  the  on  relation  to  a  state.  The  relative  distance  between  the  two 
objects  in  the  relation  is  used  to  determine  object  placement 


The  above  description  gives  an  overview  of  the  animation  process,  but  the  movement  of 
objects  relative  to  each  other  is  slightly  more  complicated.  The  previous  dependencies  of 
objects  are  also  used  to  compute  an  object's  new  location.  When  an  operator  is  applied,  the 
internal  state  of  the  planner  is  updated  and  all  the  added  and  deleted  relations  are  then  used 
to  generate  the  animation.  Deleted  relations  remove  object  dependencies  and  erase  objects 
on  die  screen;  added  relations  create  new  dependencies  among  objects  and  move  the 
objects  to  a  new  location.  Below  is  a  description  and  example  of  the  relation  effects. 

For  each  deleted  relation: 

•  Each  object  in  the  relation  is  erased  from  the  screen. 

•  Each  object’s  dependency,  based  upon  the  particular  relationship,  is  deleted. 

For  example,  deleting  the  (holding  b  An_Amj  relation  erases  the  objects  from 
the  screen  and  breaks  the  dependencies  cfB  and  An  Arm. 

For  each  added  relation: 

•  New  dependencies  between  objects  are  recorded  based  upon  the  relation.  For 
example,  when  the  relation  (on  B  A)  is  added.  Bis  set  to  depend  on  A. 

•  The  order  of  object  movement  (i.e.,  move  an  object  before  another  object)  is  set 
based  upon  the  relation.  For  example,  in  the  previous  on  relation,  A  is  moved 
before B. 

•  A  relative  displacement  function  is  computed  for  each  object  For  example,  the 
function  (movm-rol  o  -30  a  b)  is  stored  for  the  object  B‘s  move  junction  relative 
to  A. 

•  A  topological  sort  on  die  objects  is  done  based  upon  its  previous  and  present 
dependencies.  For  example,  in  the  states  of  Figure  320  sort  all  objects  in  the  state 
(A,  B,  C,  An_Arm,  Affable). 

•  Move  all  the  objects  to  their  new  locations  in  the  order  of  the  sort  using  the 
displacement  function.  This  is  done  by  applying  the  displacement  functions  to  the 
objects  in  sorted  order. 

•  Redraw  all  the  objects  in  the  domain. 

The  deletion  of  relations  must  be  done  before  the  addition  of  relations.  Using  the  above 
algorithm  in  all  the  domains  that  were  developed  in  APPRENTICE,  the  solution  trace 
provided  a  realistic  animation  of  the  planner  execution.  In  cases  where  there  are  circular 
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dependencies  this  animation  may  break  down  but  this  did  not  become  a  factor  in  the 
Hnmainn  developed  thus  far. 


323  Primitive  Elements  for  the  Domain  Builder 

APPRENTICE  is  just  one  implementation  of  domain  developing  techniques  that  aid  in 
knowledge  acquisition.  This  section  removes  the  implementation  details  and  describes  the 
underlying  set  of  primitives  that  are  needed  to  develop  a  domain  using  the  APPRENTICE 
technique.  Understanding  these  primitives  will  make  the  incorporation  of  newly  developed 
interface  techniques  easier.  These  primitives  support  the  development  of  domain 
definitions  that  have  objects,  operators,  states,  and  instances.  The  primitives  are  defined 
below,  followed  by  examples  from  both  the  blocks  world  and  a  business  organization  chart 

Hnirmin 

Object  Creation  -  The  ability  to  define  a  visual  representation  of  the 
domain  objects.  For  example,  visually  represent  a  block  or  a  business 
division. 

Object  Variable  Definitions  -  The  ability  to  refer  to  a  class  of  objects.  This 
is  used  in  the  definition  of  relations  and  operators.  For  example,  be  able 
to  make  reference  to  any  block  or  business  division. 

Object  Connection  Points  -  The  ability  to  define  a  set  of  physical 
locations,  either  explicitly  or  implicitly,  on  each  object  and  allow  the 
objects  to  connect  with  one  another.  This  will  form  relations  between  the 
objects.  For  example,  form  connections  at  the  top  and  bottom  of  a  block 
or  connections  at  the  top  and  bottom  of  a  division  icon. 

Object  Attributes  -  The  ability  to  associate  attributes  with  individual 
objects.  For  example,  the  weight  of  a  block,  the  number  of  workers  in  a 
division. 

Relation  Definition  -  The  ability  to  define  relationships  between  objects 
types  or  instances  by  connecting  their  connection  points.  For  example, 
block  on  another  block,  subdivision  under  a  division. 
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Relation  Graphical  Relative  Position  -  The  ability  to  define  the  graphical 
relative  position  of  objects.  This  is  important  during  animation  to  give  a 
realistic  portrayal  of  the  solution.  For  example,  a  block  positioned  on  top 
of  a  another  block,  a  subdivision  positioned  underneath  a  division. 

Relation  Dependency  *  The  ability  during  the  graphical  trace  for  objects  to 
be  drawn  relative  to  one  another.  The  ability  to  define  a  dependency  list 
defining  the  drawing  precedence  of  objects  is  also  important  This  can  be 
implicitly  or  explicitly  controlled  by  the  user.  For  example,  use  the 
location  of  the  bottom  block  to  get  the  relative  position  of  the  top  block, 
use  the  division  location  to  get  the  position  of  its  subdivision. 

Composition  Element  Creation  -  The  ability  to  create  a  static  description 
or  state  using  a  set  of  relations.  For  example,  block  A  on  block  B  and 
block  B  on  a  table,  a  business  organizational  chart. 

Transition  Element  Creation  -  The  ability  to  use  the  differences  between 
composition  elements  to  describe  a  state  transition.  For  example,  pickup  a 
block,  reassigning  a  subdivision. 

Solution  Tracing  -  The  ability  to  inspect  the  problem  solution  as  operators 
are  applied.  For  example,  animate  the  robot  arm  moving  or  animate  a 
subdivision  changing  the  division  it  reports  to  during  a  reorganization. 

The  above  primitives  can  be  combined  together  to  create  a  system  that  allows  users  to 
develop  domains  fester  and  more  accurately  (see  Chapter  4).  The  development  paradigm  is 
based  on  creating  domain  objects,  relations,  operators,  and  states.  Below  are  the  definitions 
of  these  elements. 

Prototype  and  Instance  Definition  -  A  prototype  is  a  class  of  physical 
objects  defined.  These  prototypes  are  graphically  defined,  have  ways  of 
expressing  connection  with  other  physical  objects,  and  can  represent 
attributes  about  themselves.  An  instance  is  an  actual  physical  object 
copied  from  the  prototype  definitions. 
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Relations  Definition  -  Relations  are  definitions  of  how  physical  objects 
connect  with  one  another.  The  definition  should  include  the  objects 
loaning  the  relation,  the  physical  orientation  between  die  objects,  and  die 
interconnected  dependency  between  objects.  The  latter  two  criteria  are 
used  for  the  animation  of  the  domain. 

State  Definition  -  States  are  a  set  of  objects  and  their  relations  with  each 
other.  States  are  defined  by  creating  composition  elements  previously 
discussed. 


Operator  Definition  -  Operators  are  sets  of  objects  and  relations  defining 
when  the  operator  is  applicable  to  a  situation  and  a  set  of  effects  that  occur 
when  an  operator  has  been  applied.  Therefore,  the  operator  is  a 
combination  of  the  composition  element  for  the  before  part  and  a 
transition  element  describing  the  difference  between  the  before  part  and  an 
after  state  for  the  effects  part 

Solutions  Animation  -  Problem  solving  can  be  graphically  traced  as  the 
problem  is  being  solved,  providing  visual  feedback  from  the  problem 
solver  by  using  the  relation  definitions.  An  algorithm  for  dynamically 
tracing  a  solution  was  given  in  section  3.22. 

Using  APPRENTICE  as  the  model,  the  domain  elements  described  above  are  shown  to  be 
an  effective  model  for  doing  knowledge  acquisition  of  visual  domains.  It  has  also  been 
demonstrated  that  tbese  elements  can  be  translated  into  code  that  a  planner  can  use  to  solve 
a  problem  in  the  domain.  With  these  primitives,  other  user  interfaces  can  be  built  to  be 
effective  in  providing  users  with  an  intuitive  system  for  KA. 

33  Framegraphics  -  Low  Level  Graphics 

A  flexible  graphical  tool  was  needed  to  build  an  interactive  system  like  APPRENTICE.  I 
developed  a  frame-based  graphical  system  called  Framegraphics  for  the  development  of 
APPRENTICE.  Because  of  its  ease  of  use,  flexibility,  and  extensibility,  Framegraphics  has 
also  been  used  as  the  interface  for  building  a  piano  tutoring  system  [Joseph  91]  and  an 
ontological  graphical  editor  [Nirenburg  88]. 
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This  object-oriented  graphical  toolkit  was  built  using  Framelrit  2.0  [Nyberg  88],  a  frame- 
based  knowledge  representation  language.  Frame  graphics  provides  a  machine-independent 
graphical  system  for  rapidly  prototyping  user  interfaces  in  a  Common  Lisp  environment.  It 
is  currently  running  on  the  Macintosh  under  Allegro  1.2,  on  the  IBM  RT/PC  under 
CMULisp  and  XI 1,  and  on  the  Sparc  workstation  under  Allegro  and  XI 1.  Figure  3.21 
shows  a  block  diagram  of  Framegraphics  organization. 


Framegraphics  1.0 

Machine  Independent 


Ftamelrit2.0 
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Figure  3.21:  Framegraphics  diagram. 

In  Framegraphics,  graphical  objects  are  represented  by  frames  and  stared  in  a  is-a/instance- 
of  hierarchy.  The  parent  frame  of  all  graphical  objects  is  the  frame  "graphic-object,"  but 
only  instance-of  frames  within  the  graphic-object  hierarchy  are  displayable  objects.  The 
graphic-object  frame  stores  the  default  behavior  for  graphical  objects.  This  default  behavior 
is  in  tire  form  of  functions,  demons,  and  values  stored  in  slots.  This  means  that  a  graphical 
object  can  be  created  with  very  little  work,  yet  have  default  behavior.  This  default  behavior 
also  helps  users  quickly  learn  how  to  use  the  application  system  by  providing  a  consistent 


The  system's  default  behavior  is  modelled  after  the  Macintosh  interface.  Objects  can  be 
selected  by  a  mouse  click,  they  can  be  moved  by  dick  and  drag,  multiple  selected  objects 
can  be  moved,  etc. 
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Most  of  this  default  behavior  is  implemented  as  function  names  stored  as  slot  values.  To 
run  a  particular  function,  first  the  slot  value  is  retrieved  using  inheritance  Then  t!s  „  value  is 
executed  with  the  current  frame  information  as  its  parameters.  The  move-proc  slot  is  an 
example  of  this.  When  a  screen  item  is  moved,  the  move-proc  slot  is  queried.  If  a  value  is 
not  found  locally,  which  is  usually  the  case,  the  inheritance  mechamrm  returns  a  value 
from  the  hierarchy.  This  value  is  then  used  as  a  function,  and  the  arguments  of  the  local 
frame  are  the  parameters.  This  is  a  very  powerful  method  to  share  code  among  objects. 

This  default  behavior  can  also  be  easily  modified  or  extended  for  an  individual  object  or  a 
class  of  objects,  hi  APPRENTICE  it  was  extremely  easy  to  add  a  help  facility.  It  took  only 
an  hour  to  develop  a  functional  help  system  so  that  objects  on  the  screen  could  provide 
information  about  themselves.  This  was  accomplished  by  adding  a  heip-proc  slot  with  a 
help  function  to  the  high  level  "apprentice-graphic-object"  frame,  the  top  display  frame  for 
the  apprentice  interface.  This  function  took  the  local  frame  name  and  retrieved  information 
from  its  help-mao  slot  These  strings  were  then  displayed  along  with  the  frame  name.  By 
adding  a  textual  description  to  the  selected  heip-msg  slot,  help  information  could  be 
tailored  for  individual  screen  objects  or  classes  of  objects.  Currently  the  help  system  is 
activated  by  an  alt-click  selection  on  a  particular  graphical  object  Again,  this  activation  was 
caused  by  adding  a  function  to  the  ait-ciick-proc  slot  that  called  the  heip-proc  slot 
function  of  a  frame.  This  activation  could  be  easily  changed  to  work  another  way.  For 
example,  help  could  be  activated  by  clicking  a  button  that  displays  information  about  the 
selected  frames.  Also  note  that  the  help  function  can  be  made  different  for  each  object  just 
by  changing  the  heip-proc  slot  function  for  the  different  frames. 

In  the  same  manner,  I  was  able  to  add  in  a  day  the  ability  to  generate  a  Postscript  file  from 
a  display  screen.  This  was  developed  after  most  of  the  APPRENTICE  system  was  already 
built  I  was  able  to  modify  the  whole  system  by  adding  a  function  slot  ps-proc.  In  this  slot 
each  frame  or  class  of  frames  would  have  a  function  that  wrote  to  a  file  the  Postscript 
commands  for  displaying  the  object  This  function  was  much  like  the  drawing  routine  for 
the  objects.  To  create  a  Postscript  file  of  a  window,  all  that  was  needed  was  to  loop  through 
the  display  list  and  have  each  frame  run  its  ps-proc  function  sending  the  output  to  a  file. 

As  with  any  object-oriented  system,  extending  object  types  is  a  simple  matter.  The  editor 
for  die  model  window  was  developed  and  integrated  into  die  system  in  about  a  day.  It  is 
selectable,  moveable,  and  responds  to  help  messages  consistent  with  other  objects,  yet  it 
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also  allows  the  building  of  graphical  objects.  Another  new  object  type  developed  was  an 
object  that  displays  graphs  and  allows  interaction  (selection,  movement,  and  click 
procedures)  with  the  nodes  of  that  graph.  The  development  and  integration  of  these  objects 
was  effortless  and  straightforward. 
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Chapter  4  -  Empirical  Analysis:  User 

Studies 


The  APPRENTICE  system  was  designed  as  a  knowledge  acquisition  tool  to  facilitate  the 
construction  of  Prodigy  domains.  To  be  useful,  die  system  needed  to  be  usable  by  a  wide 
range  of  people  and  provide  more  functionality  and  understandability  than  other  systems. 
Four  studies  were  done  with  APPRENTICE  to  evaluate  its  performance  with  multiple 
users  and  multiple  domains,  as  well  as  the  evolution  of  the  system  use  over  time.  The  four 
studies  are  comprised  of  a  study  of  the  system's  coverage  and  usability,  a  study  that 
compares  APPRENTICE  and  Emacs,  a  study  of  system  usage  over  time,  and  a  study  to 
build  a  medium  size  domain.  This  chapter  will  describe  each  of  these  studies  in  detail. 

4.1  Study  1:  Coverage  and  Usability 

To  test  the  coverage  and  usability  of  APPRENTICE,  32  students  from  an  advanced  AI 
class  participated  in  developing  their  own  domains  using  APPRENTICE.  The  objective  of 
this  study  was  to: 

•  Have  multiple  subjects  use  APPRENTICE.  This  provided  insight  on  how  users 
interacted  with  the  system.  It  helped  to  determine  what  is  easy  and  what  is  difficult 
for  others  to  understand.  It  also  showed  if  individual  conceptual  models  can  be 
incorporated  into  domain  building  in  APPRENTICE. 

•  Get  different  types  of  domains  built  using  APPRENTICE.  This  gave  an  indication  of 
what  types  of  domains  can  be  built  using  this  tooL  It  also  investigated  whether  the 
domain  building  philosophy  was  sound. 

•  Determine  what  additional  functionality  is  needed.  This  allowed  me  to  more  fully 
debug  and  enhance  the  system,  because  other  users  invariably  tried  things  that  I  had 
not  considered.  &  also  pointed  out  additional  functionality  needed. 

•  Measure  ease  of  building  a  working  domain  from  a  concept  using  APPRENTICE. 
This  allowed  me  to  quantify  the  tune  it  takes  a  person  unfamiliar  with 
APPRENTICE  to  use  the  system  productively. 
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To  reach  die  above  objectives,  each  subject  built  in  APPRENTICE  a  domain  that  they 
specified.  The  time  to  build  the  domain  and  the  final  domain  produced  was  recorded.  What 
follows  is  a  description  of  die  building  process,  examples  of  two  different  subjects' 
domains,  and  the  measured  results  and  analysis  for  all  subjects. 

4.1  J.  Study  1:  Hypothesis 

The  majority  of  users  in  this  study  should  be  able  to  use  APPRENTICE  successfully  to 
build  new  domains. 

4.12  Study  1:  Procedure 

The  participants  were  students  enrolled  in  the  class  Artificial  Intelligence: 
Representation  and  Problem  Solving.  The  class  description  from  the  University 
catalogue  is  as  follows: 

Artificial  Intelligence:  Representation  and  Problem  Saivint 
Intelligent  computer  programs  can  solve  problems,  understand  natural  language,  reaso.i  about 
their  actions,  and  learn  from  experience.  They  do  these  things  by  manipulating  internal  symbolic 
representation.  The  course  will  cover  the  main  types  of  symbolic  knowledge  representation  and 
the  main  techniques  for  planning  and  problem  solving.  LISP,  a  computer  language  designed  for 
symbolic  programming,  will  be  taught  during  the  course,  and  there  will  be  a  required 
programming  project  in  LISP. 


Each  student  was  required  to  develop  a  simple  domain  on  paper  using  a  STRIPS  type 
syntax  [Fikes  71]  as  part  of  their  normal  course  work.  The  students  knew  nothing  about 
APPRENTICE  while  they  were  developing  these  domains.  During  die  class  after  the 
students  finished  the  above  assignment,  I  gave  a  forty-five  minute  lecture  on 
APPRENTICE.  As  an  optional  assignment,  the  students  were  asked  to  encode  their 
domain  in  APPRENTICE.  Thirty-two  people  volunteered  to  do  the  assignment  Each 
student  individually  did  the  following  procedure: 


Preliminary  Setnt, 

1)  The  student  was  given  a  three-page  description  of  APPRENTICE  to  read.  This 
description  is  listed  in  the  Appendix  D  -  Apprentice  Description. 

2)  I  gave  a  short  introduction  describing  domain  elements  (objects,  relations,  and 
operators). 

3)  The  student  wrote  down  the  objects,  relations,  and  operators  in  their  domain. 

4)  The  student  was  given  a  brief  demonstration  of  how  to  build  a  simple  domain  (the 
blocks  world)  in  APPRENTICE. 

This  entire  preliminary  setup  took  approximately  40  minutes. 
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Tea  Procedure 

5)  The  students  independently  built  their  domains  using  APPRENTICE. 

During  the  study,  the  students  never  referenced  their  previous  paper-based  assignment 

With  only  two  exceptions,  all  students  were  able  to  build  their  domains  in  APPRENTICE. 
One  student  (subject  3)  needed  an  additional  function  for  his  domain  that  the  system  did 
not  support  at  die  time;  another  student's  paper  domain  (subject  4)  contained  no  search,  so 
he  built  another  domain. 

The  students'  domains  fall  into  several  categories:  recipes  (making  carrot  cake,  Kool-Aid, 
etc.),  transportation  (moving  blocks  outside  the  house,  delivering  pizza,  etc.),  games  (eight 
puzzle),  controlling  an  appliance  (playing  a  CD  player,  washing  clothes,  etc.),  and  Biology 
(building  DNA  molecules).  The  students'  domains  ranged  in  complexity  from  the  eight 
puzzle  domain  (containing  2  objects,  3  relations,  and  1  operator)  to  a  domain  to  compose  a 
sub-part  of  a  DNA  molecule  (containing  10  objects,  14  relations,  and  11  operators).  The 
development  time  far  die  domains  ranged  from  1  hour  to  4.5  hours.  The  mean  time  for 
domain  completion  was  2  hours  and  4  minutes.  For  a  description  of  the  entire  subject 
population  and  domains  built,  see  Appendix  A. 

4.13  Study  1:  Results 

The  next  section  describes  two  representative  domains  (the  eight  puzzle  and  the  DNA 
molecule  domains)  along  with  the  students'  visual  representations  and  the  system- 
generated  code. 

4J3J  Eight  Punk  Domain 

The  eight  puzzle  domain  consists  of  squares  adjacent  to  one  another  and  numbered  tiles 
that  sit  on  top  of  the  squares.  This  was  the  simplest  domain  built  as  part  of  study  1.  The 
playing  board  is  composed  of  3x3  squares  and  a  set  of  tiles  on  all  but  one  of  the  squares 
(there  is  always  a  square  without  a  tile  on  it).  The  tiles  have  unique  identifiers  (e.g. 
numbers,  letters,  etc.).  A  tile  can  be  moved  to  the  empty  square  if  the  tiled  square  is 
adjacent  to  the  empty  square.  The  tiles  start  out  in  one  configuration;  the  goal  is  to  get  to  a 
final  configuration  through  a  series  of  tile  moves.  Figure  4.1  illustrates  the  eight  puzzle 
game. 


Page  47 


1 

««■*■>•$•  t '  ? 

3 

y 

W" 

4 

2 

5 

a* 

S*  1 

7 

8 

6 

Initial  stata 


Move  2  up  to  Sq2 
Move  5  over  to  Sq5 
Move  6  up  to  Sq6 

Xntaraadiata  Moves 


* 

HJ 

1 

2 

3 

S* 

Ht 

a# 

4 

5 

6 

v 

S# 

7 

8 

rinal/Ooal  Stats 


Figure  4.1:  Illustration  of  eight  puzzle  game:  Initial  and  final  state  along  with  intermediate 

moves. 


Subject  8  encoded  the  eight  puzzle  domain  in  APPRENTICE  as  follows.  The  subject  first 
built  two  objects  (a  tile  and  a  square)  with  connection  points.  The  tile  and  the  square  are 
pictured  in  Figure  4.2. 


<Square> 


x 

<Tile> 


Figure  4.2:  Square  and  tile  objects  for  the  eight  puzzle  domain. 


The  next  step  was  to  build  relations.  There  are  three  relations.  They  are  "adjacent,"  one 
square  connected  to  another  "on,"  a  square  with  a  tile  on  it;  and  "square-empty,"  a  square 
without  a  tile  on  it  The  three  relations  are  shown  in  Figure  4.3.  Note:  die  box  on  the  square 
connection  point  in  the  square-empty  relation  means  that  the  relationship  is  true  if  the 
connection  does  not  exist  (there  is  no  connection  between  the  square  and  any  die). 


On 


Adjacent 

Figure  4J:  Relations  for  the  eight  puzzle  domain. 


The  only  operator  is  to  move  a  die.  This  operator  is  pictured  in  Figure  4.4.  The 
automatically  generated  code  that  the  system  produced  is  in  Figure  4.5.  The  MOVE 
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operator  requires  that  there  exist  two  squares  that  are  adjacent  to  one  another,  and  a  tile  is 
on  one  square,  and  the  adjacent  square  is  empty.  When  the  operator  fires,  die  die  moves 
from  the  initially  died  square  to  the  other  square,  leaving  the  previously  died  square  empty. 


Nera«:  Mov« 
DONE 


<dqu*J< 


•23  l> 


<Squar«239* 


B«Cor« 


<Square238>| 


<^quaj«23l>| 

<Tilel7pJ 


After 


Figure  44:  APPRENTICE  definition  of  the  move  operator  for  the  eight  puzzle  domain. 


(Operator  MOVE 

(prams  ( (<Square238>  SQUARE) 

(<Square239>  SQUARE) 

(<Tilel78>  TILE) ) ) 

(preconds 

(and  (On  <Tilel78>  <Square238>) 

(Square-empty  <Square239>) 

(Adjacent  <Square238>  <Square239>) 

(Adjacent  <Square239>  <Square238>) ) ) 

(effects  ((del  (On  <Tilel78>  <Square238>) ) 

(del  (Square -empty  <Square239>) ) 

(add  (On  <Tilel78>  <Square239>) ) 

(add  (Square-empty  <Square238>) ) ) ) 

Figure  45:  Automatically  generated  Prodigy  code  from  the  graphically 

defined  move  operator. 


Initial  and  goal  states  were  also  built  For  brevity  of  explanation,  a  2x2  puzzle  will  be  used 
in  the  following  description.  Figure  46  shows  an  initial  state  that  the  user  defined  using  a 
2x2  puzzle  (this  problem  is  also  known  as  the  three-puzzle). 
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State 

(On  ona  Sq2) 

(On  thraa  Sq3 ) 

(On  two  Sq4) 
(Square- empty  Sq2) 
(Adjacent  Sql  Sq2) 
(Adjacent  Sq2  Sql) 
(Adjacent  Sq2  Sq4) 
(Adjacent  Sq4  Sq2) 
(Adjacent  Sq4  Sq3) 
(Adjacent  Sq3  Sq4) 
(Adjacent  Sql  Sq3) 
(Adjacent  Sq3  Sql) 

AeCiHfilly  Generated 
Prodigy  Slate  Definition 


Figure  4 j6:  An  initial  stale  for  the  three-puzzle  in  APPRENTICE  and  in  Prodigy. 


Apprentice  Definition 


A  goal  stale  for  the  three-puzzle  is  represented  in  Figure  4.7. 


State 

(On  one  Sql) 

(On  three  Sq3 ) 

(On  two  Sq2) 

( Square -empty  Sq4) 
(Adjacent  Sql  Sq2) 
(Adjacent  Sq2  Sql) 
(Adjacent  Sq2  Sq4) 
(Adjacent  Sq4  Sq2) 
(Adjacent  Sq4  Sq3) * 
(Adjacent  Sq3  Sq4) 
(Adjacent  Sql  Sq3) 
(Adjacent  Sq3  Sql) 

Artorealkadly  Generated 
Prodigy  State  Definitioa 


Figure  4.7:  A  goal  state  for  the  three-puzzle  in  APPRENTICE  and  in  Prodigy. 


By  specifying  tire  initial  state,  goal  state,  and  operators),  the  problem  can  now  be  run  with 
animation.  Figure  4.8  shows  a  set  of  snapshots  along  the  solution  path.  Since  this  was  an 
easy  problem,  die  solution  was  found  right  away. 


Apprentice  Definition 


Page  SO 


Initial  Goal 


Mov*  on*  to  Sql 


Move  two  to  Sq2 
Goal  State  Achieved 


Figure  4.8:  Steps  for  solving  the  3-puzzle. 


Another  domain  that  a  student  built  was  a  procedure  for  building  sub-parts  of  a  DNA 
molecule  with  organic  building  blocks.  This  is  one  of  the  most  complex  domains  build  as  a 
part  of  study  1.  This  domain  is  a  student's  view  of  a  very  complex  process  and  should  not 
be  viewed  as  a  valid  description  of  a  genetic  process. 


DNA  is  a  double  helical,  long  molecule  structure  composed  of  several  building  blocks.  The 
connections  between  die  two  parallel  sides  consist  of  pairs  of  organic  base  elements: 
adenine,  thymine,  guanine,  and  cytosine.  Adenine  can  only  pair  with  thymine,  and  guanine 
can  only  pair  with  cytosine.  This  domain  deals  with  the  construction  of  these  nucleotide 
pairs. 


The  basic  elements  in  this  domain  are  the  organic  building  blocks  (cytosine,  guanine, 
thymine,  adenine),  a  chemical  mediator  (enzyme),  a  transport  element  (ribosome),  and 
nucleotide  pairs  (art-pairs  and  g-c -pairs).  These  objects  are  shown  below. 
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Figure  4dfc  Objects  in  the  DNA  molecule  domain. 


The  relations  in  this  domain  are  depicted  below  and  represent  the  interaction  between  the 
domain  objects.  These  relationships  are  between  the  organic  building  blocks,  the  ribosome 
and  the  organic  building  blocks,  the  enzyme  and  the  organic  building  blocks,  and  the 
composition  elements  of  the  nucleotide  pairs. 


ribo-holding  ribo-holding  ribo -holding  ribo-holding 

-guan  -thy*  -cyto  -adan 


guan-n*xt-to-cyto  adan-naxt-to-ehym  enzy-naxt-to-aden 


g-and-c-part-of-gc  a-and-t-part-of-at  enzy-next-to-guan 


Figure  4.10:  Relations  in  the  DNA  molecule  domain. 

Finally,  the  operators  consists  of  the  procedure  to  connect  the  organic  building  blocks  into 
the  appropriate  nucleotide  pair.  This  consists  of  using  the  ribosome  to  transport  the 
building  blocks  to  the  correct  location  and  allow  the  elements  to  be  connected  together. 
Below  are  the  visual  representations  of  the  operators.  The  generated  code  will  be  included 
in  Appendix  B. 
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Pick-up- cyto 


Pick-up-guan 


Pick-up-enzy 


Mak«-g-c-p«ir  Mak«-a-t-pair 

Figure  4.11:  Operators  in  the  DNA  molecule  domain. 


Finally  the  initial  and  goal  state  of  a  problem  is  shown.  The  system  then  figures  out  the 
sequence  of  operators  to  apply  to  go  from  this  initial  state  to  the  goal  state. 


Figure  4A2:  Initial  problem  state  in  the  DNA  molecule  domain 


Figure  4.13:  Goal  problem  state  in  the  DNA  molecule  domain. 


The  solution  for  the  above  problem  is: 


.  pick-up-guan  guanl  ribol 
put-g-naxt-to-c  guanl  ribol  cycol 
pick-up-«nzy  ribol  anzyl 
put-a-naxt-to-g  ribol  guanl  anzyl 
■aka-g-c-pair  g-c-pairl  cytol  guanl  anzyl 
pick-up-adan  ribol  adanl 
put-a-naxt-to-t  adanl  thyul  ribol 
pick-up-anzy  ribol  anzy2 
put-a-naxt-to-a  ribol  anzyl  adanl 
aaka-a-t-pair  thyal  adanl  anzyl  a-t-pairl  thynl 


After  finish^ri  their  domains,  they  were  asked  to  complete  a  questionnaire.  Their 

overall  impressions  were  very  favorable,  and  their  comments  helped  me  to  improve  the 


system.  On  the  following  page  is  a  representative  sample  of  these  comments. 
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Excerpt  of  Quotes  from  Subjects  for  Study  1 


I  had  a  aaad  fix  multiple  robdaesas  bat  throe  wm  no  easy  wey 
to  do  this. 

Ths  shinty  to  specify  objects  befog  a  robcfoas  a t  uMber 
objMtbnrinfliUi.' 

Sadi  a  system  Maas  necessary  in  ante  to  deliver  the  power 
of  planning  engines  to  compote-laypeopfe. 

Robot  Taking  Customer's  Order  - 1 


Tha  interactive  nature  of  the  Apprentice 
system  made  tha  development  of  the  do¬ 
main  aid  tha  solution  generation  of  the 
problems  fan. 

Bicycling -& 


The  graphical  interface  is  very  straightforward  to  use  and  sim- 
piy  requires  a  brief  session  of  just  playing  around  with  the 
system  to  get  used  to  it. 

People  Showering  -  2 

Certain  functionality  of  the  system  and  its  use  was 
underspecified  in  the  system’s  help  feature  and  in 
the  hardcopy  documentation. 

A  more  thorough  manual  has  been  written  and 
tha  help  system  has  been  updated.* 

Killing  Roach  -  7 


For  all  the  praise  I  offered  in  the  above  paragraph. 
I  must  now  urge  you  to  make  the  user  interface  even 
easier,  and  more  conforming  to  various  standards 
that  have  evolved  in  UX  design.  For  example,  a  co¬ 
herent  and  consistent  way  to  navigate  the  program's 
states  is  needed.  Sometimes,  I  would  have  to  click 
on  a  word  to  return,  and  others  1  would  click  on  a 
button.  1  really  do  feel  like  I  am  being  nitpicky,  how¬ 
ever.  because  1  was  very  pleased  with  all  the  less 
trivial  aspect!  of  Apprentice,  in  particular  how  it  rep¬ 
resents  the  domain  with  objects,  operators  and  t 
More  masHery  ie  added  to  how  Apprentice 


Actually  drawing  the  piettre  and 
making  connections  between 
them  was  easier  than  defining  op¬ 
erators,  states  and  asrortkns  on 
paper. 

Cat  Getting  Fish  - 14 


I  built  the  domain  I  intended.  I 
even  ended  up  budding  nun  than 
I  intended. 

Moveworld  with  Blocks  -  6 


I  was  able  to  build  my  “DNA  World”  do¬ 
main  and  it  worked!  It  took  cy¬ 

tosine,  guanine,  and  thymine  molecules  and 
made  A-T  and  G-C  pairs  out  of  them  with 
the  help  of  a  ribosome  and  enzymes. 
Apprentice  was  tun  to  use.  I  was  aide  to  get 
my  domain,  which  was  just  ideas  on  paper 
to  actually  run. 

DNA  World  - 19 


I  am  akepticai  about  the  naefiilneas  of  Apprentice  (or 
even  a  more  complex  system  of  its  nature)  for  a 
mote  complicated  domain  than  the  sort  we  have 
bean  looking  at. 

This  problem  ie  dealt  with  In  the  medium  tea 


Robot  Fetching  - 10 


I  am  not  sure  about  the  scalability  issue  —  I  think  one  of  the 
bigger  problems  would  be  the  graphical  display  of  a  lot  of 
data  and  some  automated  methods  for  keeping  the  pictures 
cohesive  and  understandable. 

This  problem  la  dealt  with  In  tha  medta 
+  Add  the  ability  to  animate  the  plan  solution. 

+  Have  ability  to  save  other  plan  solution 
+  Have  the  intermediate  steps  displayed  in  an  out  put 
Passing  Blocks  - 13 


There  was  too  much  overlapping  of  graphics  and  text  Its  was  at  times 
difficult  to  tell  what  was  on  the  screen.  The  fact  that  all  user-defined 
objects  and  states  automatically  went  to  the  same  screen  location  (first 
position  on  the  left  side)  was  confusing.  The  arbitrariness  of  the  loca¬ 
tion  of  various  pieces  of  information  was  noticeable  and  added  to  the 
catufumon  of  a  novice. 

Thie  is  a  confusing  point  but  is  jut  a  programming  fix.  It  was 

I  think  that  the  state  information  (which  was  not  printed  in  novice  mode) 
is  quite  helpful.  However,  once  again,  the  screen  clutter  is  a  problem. 
Overall,  I  found  it  to  be  a  great  visual  method  for  entering  my  domain, 
and  I  agree  this  is  a  superior  method  for  knowledge  acquisition. 

Playing  CD  Player  - 12 


I  built  the  domain  I  started  out  with  although  I  added  a 
few  more  operators  and  statm  to  make  it  more  complicated. 
It  was  generally  easy  to  use.  rod  it  was  kind  of  fun.  I  liked 
the  feet  that  we  had  to  drew  our  operators  and  states.  I  think 
this  makes  it  more  interesting  and  easier  to  follow. 

Fixing  Kool-Aid  - 17 


Providing  for  multiple  objects  of  the  same  class,  but  with  different  attributes, 
or  anything  involving  attribute  was  not  as  obvious,  ft  needs  to  provide  an 
easier  method  for  dealing  with  attribute  perhaps  labeling  objects  in  rela¬ 
tions  and  operators  with  the  attribute. 

The  eMBty  to  tewlls  attribute  hater  wffl  either  be  dteaas ed  or  he- 

PfwVWe 

Playing  CD  Player  -  20 


I  .turning  whether  to  right  dick,  left  click,  etc. 
at  each  point  took  some  getting  used  to.  but 
after  time  became  easy  to  use. 

Feeding  the  Cat  -  21 


*  The  BoM  Text  is  my  comments. 
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4J.4  Study  1:  Analysis 

APPRENTICE  was  used  successfully  by  a  varied  set  of  people,  over  several  type*;  of 
domains,  in  a  relatively  short  time.  All  domains  that  the  students  tried  to  build  in 
APPRENTICE  successfully  lead  to  a  running  system.  The  students  were  diverse,  ranging 
from  college  sophomores  to  first  year  graduate  students,  and  majors  ranging  from 
Computer  Science  to  Music.  The  system  supported  the  development  of  several  types  of 
domain*  like  solving  the  eight  puzzle,  building  a  DNA  molecule,  cooking  a  carrot  cake, 
and  defrvexing  pizza. 

The  students  also  helped  debug  and  enhance  the  system.  Some  developments  that  resulted 
from  this  study  were  better  consistency  checking,  increased  functionality  with  class 
specification  and  hierarchy  of  objects,  and  increased  system-directed  help  and  user 
warnings.  Finally,  an  intangible  result  was  that  the  students  enjoyed  using  the  system.  They 
found  it  "easy  and  fun.” 

A  closer  analysis  of  the  study  1  data  provides  more  characterization  of  the  domain 
development  process.  The  average  time  for  domain  development  was  two  hours  and  four 
min.  In  Figure  4.14  statistics  from  the  data  are  shown.  Figure  4.15  is  a  plot  of  die  subjects 
versus  their  development  time.  It  also  shows  the  development  time  median  (120),  quartiies 
(100,  140)  and  whiskers  (60,  200).  Notice  that  in  the  chart  there  are  three  subject 
development  times  that  seemed  to  be  inconsistent  with  the  rest  A  light  gray  box  surrounds 
these  times.  Analysis  was  done  on  die  data  with  and  without  these  subjects. 


32.Sflbt«a 

29  Subjects 

Median  Time 

120.0 

120.0 

Mean  Time 

124.4 

114.5 

Standard  Deviation  Time 

40.7 

24.9 

Mean  Objects 

5.7 

5.9 

Mean  Relations 

&8 

8.4 

Mean  Operators 

6.1 

6.0 

Figure  4.14:  Statistics  for  32  and  29  subjects  in  study  1. 
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Figure  4.15:  Plot  of  subjects  versus  domain  development  times  from  study  1. 


I  also  analyze  die  relationships  between  the  different  elements  of  the  domains.  Figure  4.16 
shows  the  comparison  plots.  The  object  and  relation  elements  are  strongly  correlated, 
where  the  operators  are  not  correlated  with  either  the  objects  or  the  relations. 


Figure  4.16:  Comparison  plot  of  Study  1  domain  elements  with  the  29  subjects. 


Using  the  data  from  the  study  I  did  a  multiple  regression  on  the  29  subjects,  lime  was  the 
dependent  variable  and  objects,  relations,  and  operators  were  the  independent  variables. 
Also  because  of  the  close  correlation  between  objects  and  relations,  I  did  another  multiple 
regression  using  just  objects  and  operators  as  the  independent  variables.  For  each 
regression  the  squared  multiple  R  was  .2  which  means  that  other  factors  were  dominating 
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the  time  development  (e^.,  background  of  students,  complexity  of  graphical  objects, 
domain  types  being  developed,  etc.). 

4.1Ji  Study  1:  Conclusion 

The  hypothesis  was  confirmed.  All  subjects  were  able  to  encode  working  domains  using 
APPRENTICE  and  none  of  the  domains  was  preselected  for  its  visually  oriented 
properties.  APPRENTICE  showed  a  wide  applicability  to  many  domains. 

4.2  Study  2:  APPRENTICE  Versus  Emacs  Study 

This  study  compares  the  use  of  APPRENTICE  to  build  a  domain  versus  the  use  of 
Emacs,  a  typical  text  editor  commonly  used  by  programmers,  to  build  a  domain.  The 
objective  of  this  study  was  to: 

•  Test  the  productivity  of  users  with  APPRENTICE  versus  Emacs.  This  provides 
quantitative  results  about  users'  productivity  when  building  domains  utilizing  these 
two  methods. 

•  Test  user's  comprehension  of  new  domains  represented  graphically  versus  domains 
represented  textual ty.  This  gives  an  indication  of  how  well  a  domain  is  understood 
when  it  is  represented  by  pictures  or  by  simple  textual  "if-then"  rules. 

•  Test  the  usability  of  APPRENTICE  with  a  variety  of  users  from  prodigy  experts  to 
non-programmer  users.  This  provides  a  diverse  population  of  users  for 
APPRENTICE.  The  system  was  tested  and  enhanced  similar  to  study  1. 

This  is  a  two-phase  study.  Phase  1  compares  the  time  it  takes  different  types  of  users  to 
build  domains  in  APPRENTICE  versus  Emacs.  Phase  2  measures  each  subject's  ability  to 
understand  already  built  domains,  using  multiple  choice  questions. 

42.1  Study  2:  Hypothesis 

Non-technical  users  will  encode  and  understand  domains  faster  and  with  more  accuracy 
with  APPRENTICE  than  Emacs. 

422  Study  2:  Procedure 

In  both  phases  four  types  of  people  were  used: 

PRODIGY  EXPERTS  are  graduate  students  currently  working  on  various  projects 
that  use  Prodigy.  They  have  an  in-depth  understanding  of  Prodigy  and  its  use  and 
have  used  Emacs  as  their  primary  tool  for  building  domains. 
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at  INDUSTRY  EMPLOYEES  are  members  of  Carnegie  Group,  Inc.,  an  AI 
company  located  primarily  in  Pittsburgh.  These  employees  are  familiar  with  AI 
terminology  and  have  worked  with  Emacs  extensively.  They  knew  nothing  about 
Prodigy  before  the  experiment 

CMU  AI  EXPERTS  are  subjects  who  are  working  in  AI  at  Carnegie  Mellon 
University.  This  group  is  very  familiar  with  AI  programming  techniques  and  the 
use  of  Emacs  but  had  no  prior  knowledge  of  Prodigy. 

NON-TECHNTCAL  USERS  are  subjects  who  were  unfamiliar  with  the  field  of  AI 
and  hftri  only  limited  computer  experience.  Computer  exposure  for  these 
individuals  has  been  limited  to  word  processing  software,  drawing  packages,  and 
simple  database  use. 

422.1  Phasf  i :  Pmain  BxMag 

For  Phase  1  subjects  task  was  to  build  a  domain  in  Apprentice  and  a  different  domain 
in  Emacs.  The  specified  domains  were  a  package  delivery  domain  and  a  robot  path 
planning  rinmain  The  two  domains  were  of  equal  complexity.  They  used  a  similar  number 
of  objects,  relations,  and  operators.  To  gain  experience  developing  domains  in  both 
systems,  each  subject  used  a  small  pizza-delivery  domain  for  practice.  A  description  of 
each  domain  is  in  Appendix  D. 

Emacs  was  chosen  as  the  text  editor  for  development  comparison  because  it  is  currently 
used  to  develop  domains  in  Prodigy  and  is  similar  to  other  methods  used  to  build  domains 
in  typical  planning  systems.  The  Prodigy  language  syntax  has  an  if-then  syntax,  indicative 
of  the  syntax  of  a  lot  of  expert  systems.  The  instruction  that  each  user  was  given  for  how  to 
use  each  system  is  in  Appendix  D.  The  instruction  for  using  Emacs  shows  the  PDL 
without  universal  quantifiers. 

Phase  1:  PROCEDURE 

Each  subject  was  given  an  introductory  page  about  Prodigy  and  Expert  Systems  (see 
Appendix  D).  The  page  also  presented  the  very  simple  practice  domain,  a  pizza-delivery 
domain.  Once  subjects  finished  reading  the  introductory  page,  they  used  the  following 
procedure  with  each  system.  For  example,  a  subject  would  do  the  following  procedure  in 
Emacs,  and  then  repeat  the  procedure  using  APPRENTICE. 


Preliminary  Setiro 

1 )  The  subject  read  a  two-page  description  of  foe  system  they  were  going  to  use. 
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2)  The  subject  was  given  a  demonstration  building  a  subset  of  the  blocks  world  domain 

in  the  current  system. 

3)  The  subject  built  die  practice  pizza-delivery  domain  in  the  current  system. 

Erocsdmc 

4)  Finally,  the  subject  built  one  of  the  experimental  domains  on  their  own  using  the 
appropriate  system. 

After  die  first  system  was  used,  die  subjects  went  back  to  the  Preliminary  Setup  and 
repeated  die  process  with  the  other  system. 

While  the  subjects  were  building  their  domains,  timing  data  was  being  recorded.  Each 
subject  built  one  domain  using  APPRENTICE  and  the  other  domain  using  Emacs.  Each 
domain  was  built  with  each  system  being  used  by  at  least  one  person  in  a  particular  group. 
For  example,  in  the  AI  Employee  group.  Subject  1  built  the  Strips  domain  using 
APPRENTICE  and  die  Logistics  domain  using  Emacs,  whereas  Subject  2  built  the  Strips 
domain  using  Emacs  and  the  Logistics  domain  using  APPRENTICE.  This  information  is 
presented  in  Figure  4.17. 

4.2.U.1  Phase  1;  Results 

The  charts  in  Figure  4.17  display  the  amount  of  time  it  took  each  subject  to  build  a 
particular  domain  with  a  specific  system.  Subjects  are  grouped  together  according  to  the 
four  types  discussed  earlier.  Each  subject's  results  are  depicted  in  the  boxes  under  S#.  The 
domain  that  was  being  built  is  shown  along  the  left  side  of  the  box  (except  for  the  non¬ 
technical  users).  The  interior  squares  show  the  system  that  the  subject  used  to  build  the 
particular  domain  and  die  amount  of  time  it  took  that  person  to  do  it  The  order  in  which 
die  domains  were  built  is  represented  from  top  to  bottom.  The  shaded  boxes  are  the 
domains  that  were  built  using  APPRENTICE,  and  die  bold  face  type  shows  the  faster  time 
for  each  subject  The  XX  represents  an  individual's  inability  to  come  close  to  task 
completion  within  a  two-hour  period. 
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Figure  4J.7:  Phase  1  domain  building  time.  The  chart  shows  faster  development  time  for 
APPRENTICE  than  Emacs  for  all  but  the  seasoned  Prodigy  expert. 

4JL1.1.2  Phase  1;  Analysis 

Except  for  the  Prodigy  experts,  all  subjects  were  able  to  build  domains  faster  by  using 
APPRENTICE  than  by  using  Emacs.  Several  people  were  unable  to  even  build  domains 
using  Emacs.  As  expected,  the  nontechnical  users  had  the  biggest  ratio  difference  between 
domains  built  using  APPRENTICE  and  domains  built  using  Emacs,  assominfigune4.18. 
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Figure  4.18:  Ratio  of  domain  building  time  (P^/ap). 
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This  strong  indication  that  visual  domains  are  built  faster  in  APPRENTICE  than  in  Emacs 
can  be  further  analyzed.  For  this  analysis  let's  ignore  the  Prodigy  experts.  They  are  well 
versed  in  Prodigy  and  had  built  many  domains  in  Emacs  similar  to  the  domains  in  the 
study.  Ignoring  the  Prodigy  experts,  all  other  subjects  developed  domains  faster  in 
APPRENTICE.  The  probability  of  all  seven  subjects  randomly  developing  domains  in 
APPRENTICE  faster  is  1/27  or  0.78%.  This  gives  a  strong  indication  that  people  similar  to 
die  ones  in  this  study  (excluding  the  Prodigy  experts)  will  develop  visual  domains  faster  in 
APPRENTICE  than  in  Emacs. 

4222  Phase  2:  Domain  Understanding 

Phase  2  tested  the  ability  of  a  user  to  understand  a  new  domain  represented  graphically 
versus  a  domain  represented  textually.  Each  subject  was  asked  ten  questions  about  a 
domain  represented  textually,  and  ten  questions  about  a  different  domain  represented 
graphically.  The  two  domains  used  for  this  experiment  were  the  monkey  and  banana 
domain  and  the  blocks  world  domain.  As  with  phase  1,  each  domain  was  studied  using 
both  systems  far  each  subject  type. 

Phase  2:  PROCEDURE 

This  process  was  also  automated.  The  user  was  presented  with  an  operator  or  a  set  of  initial 
and  goal  states.  The  user  was  then  asked  several  multiple-choice  questions  pertaining  to  the 
representation.  Below  are  typical  graphical  and  textual  displays,  along  with  a  sample 
multiple-choice  question.  All  questions  for  both  representations  are  in  Appendix  E 


Is  operator  op4  representing? 

1)  Grabbing  the  banana  3)  Monkey  moving  the  block 

2)  Monkey  moving  himself  4)  Monkey  getting  on  the  block 


Figure  4.19:  Graphical  representations  with  a  multiple-choice  question. 
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Opl 

If 

(monkey-at-loc  <11>  <monk>) 
(connected  <11>  <12>) 

(connected  <12>  <11>) ) ) 

Then 

(del  (monkey-at-loc  <11>  <monk>) ) 
(add  (monkey-at-loc  <12>  <monk>) ) 


Is  operator  op 4  representing? 

1)  Grabbing  the  banana  3)  Monbey  moving  the  block 

2)  Monkey  moving  himself  4)  Monkey  getting  on  the  block 


Figure  4.20:  Textual  representations  with  a  multiple-choice  question. 


The  time  to  complete  all  the  questions  and  the  subject's  answers  were  automatically 
recorded  for  each  user. 

4-2- 1-2.1  Phase  2:  Remits 

The  results  of  the  users'  responses  are  shown  in  Figure  4.21.  The  highlighted  areas  depict 
answe  s  about  the  graphical  representation.  Bold  letters  represent  answers  that  were  wrong. 
Inside  the  squares  are  the  domain  and  representation  the  subject  used,  the  time  it  took  the 
subject  to  answer  die  questions,  and  die  number  of  answers  the  subject  got  wrong.  For  a 
complete  description  of  the  questions  and  the  wrong  answers  that  were  given  see  Appendix 
E. 


Page  63 


CMU  AI  Experts  AI  Industry  Employees 


Prodigy  Experts 

SI  S2 


Figure  421:  Results  of  Phase  2,  showing  much  better  understanding  (fewer  errors)  for 

APPRENTICE  than  Emacs. 
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4AI A2  Flaac  2;  Anaiiaa 

The  subjects  understood  domains  presented  graphically  more  accurately  and  more  quickly 
than  domains  presented  textually.  The  mean  time  for  answering  all  ten  questions  for  die 
graphic  representation  was  238  seconds  with  a  standard  deviation  of  87  seconds,  but  for 
the  textual  representation  the  mean  tune  was  320  seconds  with  a  standard  deviation  of  200 
seconds. 

Also  all  the  subjects  got  the  answers  correct  for  the  graphical  representation,  except  for  one 
answer  by  a  non-technical  subject  In  question  6  of  the  monkey  and  banana  domain,  the 
move  Mock  operator  did  not  have  the  monkey  connected  to  the  block  (see  Figure  4.21). 
The  subject  didn’t  think  the  monkey  was  moving  the  block  but  just  moving  himself.  Also 
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note  that  die  same  subject  understood  the  graphical  representation  much  better  than  the 
textual  representation,  as  seen  by  the  eight  wrong  answers  about  the  textual  representation. 


Is  operator  op4  representing? 

1)  Grabbing  the  banana  3)  Monkey  moving  the  block 

2)  Monkey  moving  himself  4)  Monkey  getting  on  the  block 

Figure  4w22:  Question  6  representing  die  move- block  operator. 

There  were  more  wrong  answers  for  the  textual  representation.  Four  subjects  got  a 
combined  total  of  13  wrong  answers  for  the  textual  representative  questions.  Although  the 
questions  and  the  rules  were  very  simple,  the  subjects  still  had  difficulty  quickly 
understanding  a  totally  new  domain. 

423  Study  2:  Analysis 

The  results  of  study  2  were: 

•  Domains  were  built  faster  using  APPRENTICE  than  Emacs  for  all  but  the  most 
seasoned  Prodigy  users.  This  study  was  done  with  users  of  different  degrees  of 
computer  experience.  Even  users  who  were  proficient  in  Emacs  were  able  to  use 
APPRENTICE  faster. 

•  Users  who  had  limited  computer  expertise  were  able  to  effectively  build  domains  in 
APPRENTICE,  yet  some  where  unable  to  do  so  in  Emacs. 

•  When  given  unknown  domain  definitions,  users  understood  the  domains  better  in 
APPRENTICE -type  graphical  descriptions  than  a  textual  S TRIPS-type 
methodology. 
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•  Again,  users  found  the  system  to  be  "fun  and  easy." 


42 A  Study  2:  Conclusion 

The  hypothesis  was  confirmed.  Non- technical  (and  all  non-Prodigy)  users  were  more 
productive  with  APPRENTICE  than  Emacs.  Every  user  was  able  to  build  a  successful 
domain  using  APPRENTICE  while  some  non- technical  users  could  not  build  a  domain 
using  Emacs. 

43  Learning  Study 

In  this  study  a  subject  built  four  domains  over  time.  The  goal  of  this  study  was  to  see  the 
speed  in  which  productivity  increased.  The  subject  used  for  this  was  subject  2  from  the 
Prodigy  expert  group  from  Study  2.  Three  additional  domains  where  built  by  the  subject 
over  the  course  of  one  day.  These  domains  were  hiking  world,  loading  truck  world,  and 
robot  picking  tulip  world.  They  can  be  found  in  Appendix  G.  All  domains  were  of  similar 
complexity  to  the  strips  domain  that  was  previously  built  The  domains  had  three  objects, 
four  relations,  and  two  to  four  operators  in  them.  The  results  are  as  follows. 


Strips 

Hiking 

Load  Truck 

Subject  2 

43  min 

23  min 

33  min 

htpmi 

Figure  423:  Table  of  subject  2's  domain  building  time;  The  domains  were  built  in  order 

from  left  to  right 


As  Figure  4.23  shows,  the  user  took  less  time  to  build  the  domains  as  experience 
increased.  This  experiment  is  not  meant  as  a  scientific  study  but  as  an  indication  that 
experience  using  APPRENTICE  will  improve  development  time. 

4.4  Study  4:  Medium  Size  Domain  Building 

A  medium  size  domain  with  34  operators  was  developed  to  test  the  scalability  of 
APPRENTICE.  The  objectives  of  this  study  were  to: 

•  Determine  the  APPRENTICE  techniques  useful  in  developing  larger  domains.  As 
domains  get  larger,  how  does  the  APPRENTICE  system  aid  or  hinder  the 
development  process? 

•  Explore  additional  techniques  needed  for  development  of  larger  domains.  What  are 
some  of  the  requirements  of  domains  as  they  get  larger? 


•  Investigate  issues  of  developing  a  domain  over  time.  In  the  other  studies  the 
domains  were  developed  in  one  sitting.  What  are  some  of  the  things  that  affect  the 
creation  of  a  domain  during  multiple  development  sessions? 

In  this  study,  I  took  an  already  established  large  machining  domain  and  developed  a  sub¬ 
portion  of  it  in  APPRENTICE  over  time.  The  domain  is  described  in  the  next  section, 
followed  by  what  I  learned  from  the  experience. 

4.4.1  Study  4:  Hypothesis 

APPRENTICE  scales  up  to  larger  domains. 

4.42  Study  4:  Procedure 

The  domain  I  used  had  been  previously  developed  and  recorded  in  a  technical  report  at 
CMU  [Gil  90].  An  excerpt  from  the  abstract  of  that  report  follows: 

Much  research  is  being  done  on  the  automation  of  manufacturing  processes.  The 
planning  component  in  the  production  stage  is  very  significant,  due  to  the  variety  of 
alternative  processes,  their  complexity,  and  their  interactions.  This  document 
describes  a  specification  of  some  manufacturing  processes,  including  the  machining, 
joining,  and  finishing  of  parts.  The  atm  of  this  specification  is  not  to  be  comprehensive 
or  detailed,  but  to  present  the  AI  community  with  a  model  of  a  complex  and  realistic 
application....(Gil  90] 

Over  a  three-month  period,  working  part  time,  I  developed  a  sub-portion  of  the  machining 
domain  in  APPRENTICE.  This  required  understanding  the  machining  domain,  debugging 
and  enhancing  APPRENTICE,  and  understanding  some  subtle  features  about  Nolimit.  The 
developed  domain  consists  of  33  objects,  77  relations  and  34  operators.  The  operators 
consist  of  operations  to  drill,  saw,  plane,  polish,  and  mill  parts.  The  graphical  and  textual 
representations  of  the  domain  are  included  in  Appendix  F. 

4.43  Study  4:  Results 

Larger  domains  are  very  hard  to  build  using  any  system.  In  developing  the  machining 
domain  there  were  three  areas  of  results:  1)  ways  in  which  graphical  techniques  aided  in  the 
domain  development;  2)  additional  programming  algorithms  that  aided  in  domain 
development;  and  3)  techniques  that  were  not  programmed  but  could  provide  some 
support  for  domain  development 

The  graphical  representation  of  the  domain  provided  several  aids  to  the  domain 
development  the  first  being  that  visual  images  are  very  helpful  in  not  forgetting  relevant 
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information.  When  building  the  operator  to  saw  a  part  it  was  obvious  that  the  band  saw  bad 
to  have  a  blade  in  it  and  the  pan  bad  to  be  on  the  band  saw  table.  Also  the  graphical  images 
helped  me  to  remember  what  state  I  left  the  domain  in  from  the  last  session.  This  was  very 
useful  because  I  was  constantly  going  out  of  town  during  the  development  of  this  domain. 

As  the  domain  increased  in  size  and  complexity  the  knowledge  base  changed.  These 
changes  were  in  the  form  of  object  name  changes,  object  appearance  changes,  connection 
points  moved,  relation  definition  changes,  operator  definition  changes,  and  states  definition 
changes.  Some  of  the  consistency  checking  was  already  a  part  of  the  system;  but  as  the 
machining  domain  increased,  more  consistency  checking  and  updating  was  needed.  For 
example,  when  an  object  name  changed,  the  relation,  operator,  and  state  that  used  that 
object  had  to  be  updated.  Figure  4.24  shows  the  impact  elements  have  on  other  elements 
when  they  change. 
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Figure  424:  Table  of  the  elements  that  are  impacted  when  an  element  is  changed. 

Another  feature  in  the  system  that  proved  to  be  useful  was  being  able  to  test  sub-parts  of 
the  domain.  This  was  achieved  by  being  able  to  define  a  problem  with  a  subset  of  the 
defined  operators  and  being  able  to  individually  test  an  operator  with  user-selected 
instantiation. 

As  die  domain  got  bigger  there  were  other  features  that  I  thought  could  aid  in  the  domain 
development  One  feature  was  the  ability  to  copy  and  paste  objects.  The  second  was  the 
ability  to  abstract  the  relation  definition  or  at  least  automate  the  definition  of  relations  for 
objects  in  the  same  hierarchy.  Currently  a  relation  definition  involving  a  machine  would 
also  need  to  be  explicitly  defined  for  all  different  types  machine  (e.g.,  put-vise-on-machine 
needs  to  be  defined  for  the  machine,  drill,  planner,  and  mill  objects).  It  would  be  good  if. 
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when  a  relation  was  defined  for  a  parent  object  (i*e.,  a  machine  object),  the  system  defined 
the  other  relations  automatically,  allowing  the  user  to  change  any  that  were  incorrect 
finally,  attributes  need  to  be  handled  better.  As  the  complexity  of  die  domain  increased 
attributes  became  more  and  more  important  The  size  of  the  part  and  the  hole  position  are 
some  of  these  attributes.  Allowing  the  attributes  to  be  bandied  with  a  more  graphical 
methodology  would  help  increase  the  user's  domain  understanding. 

4.4 .4  Study  4:  Conclusion 

The  hypothesis  was  confirmed.  APPRENTICE  does  scales  up  to  larger  domains.  For 
future  work  more  investigation  can  be  done  on  scaling  up  domains  in  APPRENTICE. 
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Chapter  5  -  Domain  Characteristics 


This  chapter  discusses  the  domain  characteristics  that  are  conducive  to  efficient  domain 
development  in  APPRENTICE.  Since  APPRENTICE  provides  the  ability  to  textually  edit 
any  portion  of  a  domain,  any  domain  developed  for  Prodigy  can  be  built  using 
APPRENTICE.  There  are,  however  some  characteristics  that  are  better  suited  for 
APPRENTICE’S  graphical  form  of  knowledge  acquisition.  In  essence,  APPRENTICE 
performs  best  with  domains  in  which  the  central  part  of  the  domain  is  represented  by 
objects,  relations,  and  operators;  with  domains  that  are  highly  visual;  and  with  domains  that 
involve  procedural  tasks. 

In  Study  1,  30  of  the  32  domains  suggested  by  AI  students  prior  to  any  knowledge  of 
APPRENTICE  were  easily  acquired  via  APPRENTICE.  Therefore,  visual-orientation  is  a 
very  wide-ranging  property  for  most  domains. 

It  is  my  belief  that  knowledge  acquisition  should  be  done  with  multiple  techniques,  in 
which  the  KA  system  strives  to  closely  resemble  the  way  the  expert  thinks  about  and 
solves  problems  in  the  domain  being  developed.  It  is  important  for  the  designers  of 
knowledge  acquisition  system  to  understand  different  techniques  to  help  develop  versatile 
"hybrid”  KA  systems.  With  such  combined  systems,  the  strengths  of  each  technique  can 
be  utilized  by  selecting  the  most  appropriate  one  for  each  situation. 

To  help  identify  the  strengths  of  the  APPRENTICE  techniques,  this  chapter  addresses  the 
characteristics  of  a  domain  that  makes  the  domain  amenable  to  APPRENTICE.  I  will  also 
discuss  the  ability  of  APPRENTICE  to  develop  medium  size  domains,  as  well  as  some 
issues  of  expanding  domain  knowledge. 

5.1  Positive  Domain  Characteristics 

A  set  of  domain  characteristics  emerged  as  multiple  domains  were  built  in  APPRENTICE. 
APPRENTICE- like  techniques  are  useful  with  domains  having  the  pri  ary  characteristics 
of  1)  objects  central  to  the  domain's  representation;  2)  visual  images  corresponding  to  the 
domain  objects;  and  3)  modelling  a  procedural  task.  Other  secondary  domain 
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characteristics  that  APPRENTICE  can  handle  well  are  domains  in  which  the  objects  are 
grouped  in  a  hierarchy  and  domains  in  which  object  attributes  relate  only  to  a  particular 
object 

Domains  in  which  objects  are  central  to  the  representation  can  be  described  in  terms  of  the 
objects,  relations,  and  operators.  This  domain  description  parallels  the  development  process 
far  building  domains  in  APPRENTICE.  This  allows  the  expert  to  focus  on  the  domain 
information  and  not  on  the  system  representational  language.  This  key  characteristic  has 
been  a  predominant  characteristic  in  all  the  domains  that  have  been  built  thus  far  in 
APPRENTICE. 

Alternatively,  in  the  absence  of  physical  objects,  the  APPRENTICE  approach  works  well 
if  a  mental  description  of  the  domain  with  a  highly  visual  representation  can  be  formulated 
(e.g.  packet  switching).  This  allows  the  expert  to  create  visual  images  of  a  domain,  similar 
to  the  mental  representation  that  the  expert  already  has,  and  eliminates  some  of  the  mental 
translation  that  the  expert  has  to  do  in  order  to  develop  a  domain. 

Another  characteristic  that  APPRENTICE  supports  well  is  domain  modelling  of  a 
procedural  task  that  has  a  structured  approach  to  planning  problem  solutions,  such  as 
cooking  a  carrot  cake  or  machining  a  metal  part  These  tasks  follow  the  expert/apprentice 
paradigm  in  which  die  expert  demonstrates  to  an  apprentice  how  to  solve  a  problem  in  the 
domain.  The  expert  is  first  concerned  about  building  a  common  language  with  the 
apprentice;  thus,  object  and  relation  descriptions  are  needed.  Then  the  expert  is  concerned 
about  relaying  to  the  apprentice  the  procedural  or  operational  information  in  which  the 
objects  and  relations  are  used  to  describe  the  operational  information.  For  domains  that  are 
not  solving  a  procedural  task,  such  as  text  composition  or  unstructured  troubleshooting,  a 
different  set  of  techniques  are  needed,  and  an  APPRENTICE-like  approach  would  not 
work  welL 

The  other  characteristics  that  have  been  shown  to  be  supported  by  APPREN1  iCE  are  the 
hierarchy  relationship  between  objects  (e.g.,  drill  is-a  machine),  and  the  attribute  description 
of  individual  objects  (e.g.,  weight  of  block,  etc.). 
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5,2  APPRENTICE  Limitations 

There  are  a'io  certain  domain  characteristics  that  do  not  seem  to  be  conducive  to  the 
APPRENTICE  techniques.  Of  course,  domains  that  are  not  visual  prove  to  be  difficult  to 
express  with  the  system.  Another  limitation  in  the  current  system  is  the  difficulty  of 
expressing  multidimensional  relations.  These  types  of  relations  represent  the  relationship 
between  objects  in  two-  or  three-  dimensional  space.  I  briefly  discuss  the  needs  and 
specifications  for  allowing  this  type  of  relations  definition  in  section  6.3.3. 

Another  difficulty  with  APPRENTICE  derives  from  the  planning  system  itself.  Nolimit 
does  not  handle  infinite  type  variables  (i.e.,  numbers,  time,  etc.)  very  cleanly.  Variables  of 
this  type  cannot  be  bound  in  an  operator  from  the  state  definition  during  planning.  This 
requires  custom  Lisp  code  to  be  written  to  generate  the  appropriate  instances  of  the 
variables.  This  is  handled  by  brute  force  in  the  APPRENTICE  system,  and  none  of  the 
domains  observed  made  extensive  use  of  infinite  type  variables.  Because  the  difficulty  in 
dealing  with  infinite  types  is  due  to  the  underlying  problem  solver,  APPRENTICE  may 
deal  with  inaaite  types  better  using  another  planner. 

For  the  largest  domains  that  were  built  in  APPRENTICE,  the  limited  screen  real  estate  did 
not  cause  a  problem.  But  as  domains  get  bigger  and  there  is  a  high  degree  of  interaction 
among  objects,  I  expect  that  there  will  not  be  enough  room  on  the  screen  for  all  the 
graphics.  This  limitation  can  be  minimized  by  developing  better  techniques  for  allowing  the 
user  to  store  and  display  needed  information,  such  as  techniques  for  creating  abstract  object 
types  that  can  graphically  represent  multiple  objects  as  one.  For  example  the  strips  domain 
could  be  extended  to  have  multiple  buildings  with  several  rooms.  A  plan  consists  of 
moving  packages  from  one  location  to  another.  This  may  mean  moving  packages  between 
buildings.  To  encode  the  operator  to  move  packages  between  buildings  only  the  buildings 
need  to  be  considered.  To  encode  the  operator  to  move  packages  between  rooms  only  the 
rooms  ir  a  particular  building  need  to  be  considered.  By  allowing  abstract  object  definitions 
to  handle  this  type  of  context  sensitive  usage  screen  real  estate  can  be  conserved  and  larger 
domains  can  be  better  organized. 

Finally,  some  information  can  be  expressed  more  concisely  and  with  greater  ease  in  a  non¬ 
visual  representation  (e.g.,  mathematical  formulas,  programming,  etc.).  As  discussed 
previously,  the  expert  should  be  allowed  to  use  other  methods  to  represented  domain 
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information,  thus  allowing  maximum  flexibility  for  the  expert  APPRENTICE  shows 
some  amount  of  this  ability  by  allowing  the  generated  code  to  be  edited  manually. 

53  Techniques  That  Aid  Large  Domain  Development 

Large  domains  share  certain  characteristics  that  have  to  be  addressed.  With  large  domains 
there  are  many  things  to  keep  track  of,  multiple  interactions  to  coordinate,  and  many  details 
to  incorporate  into  the  knowledge  base.  Several  APPRENTICE  techniques  aid  in  the 
development  of  these  large  domains. 

The  ability  to  have  the  visual  representation  closely  match  the  physical  domain  aids  in 
cueing  the  expert  to  needed  information  during  domain  development  For  example,  if  the 
expert  is  developing  the  operator  to  drill  a  hole  in  a  part  it  is  obviously  easy  to  graphically 
identify  the  drill-bit  not  being  in  place.  This  error  would  be  more  difficult  to  find  in  a 
textual  representation.  As  the  domain  increases  in  size,  this  ability  to  detect  obvious 
inconsistences  becomes  more  and  more  valuable. 

The  structure  of  APPRENTICE  gives  an  expert  a  consistent  paradigm  for  developing  a 
knowledge  base,  while  also  allowing  the  flexibility  to  build  sub-parts  one  at  a  time. 
Because  the  domain  closely  resembles  the  knowledge  being  encoded  and  the  development 
procedures  are  easily  understood,  die  expert  has  a  clear  focus  on  what  information  needs  to 
be  added  and  where  that  information  should  go.  Also,  the  expert  can  develop  different  parts 
of  the  domain  as  appropriate.  This  allows  the  expert  to  change  focus  and  develop  relevant 
information.  For  example,  the  expert  can  develop  a  portion  of  the  domain  (e.g.,  relevant 
objects,  relations,  and  operators)  to  drill  a  hole  in  a  part,  and  later  develop  the  portion  of  the 
domain  to  polish  a  part  This  flexibility  is  very  important  in  developing  large  domains  in 
segments. 

Large  domains  are  developed  over  time  during  many  sessions.  The  expert  may  go  for  long 
periods  of  time  without  using  or  reviewing  certain  information.  The  visualization  helps  the 
expert  quickly  recall  the  previous  work.  The  graphics  also  help  experts  at  the  beginning  of  a 
new  session  determine  where  they  ended  from  the  last  session. 

With  a  shared  visual  representation,  multiple  developers  can  easily  communicate  to 
develop  and  maintain  a  large  knowledge  base.  The  demonstrated  ease  in  domain 
understanding  indicates  the  feasibility  of  this  coordinated  effort 
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As  a  domain  increases  in  size  during  development,  some  of  the  APPRENTICE  techniques 
can  be  used  to  help  make  this  development  more  efficient  APPRENTICE  has  several 
capabilities  that  aid  in  the  expansion  of  a  domain.  These  are  consistency  checking, 
inheritance  of  objects,  unknown  relations  warnings,  and  a  customizable  interface. 

Consistency  checking  allows  changes  made  to  parts  of  a  domain  to  be  propagated  to  the 
other  relevant  parts  of  the  domain.  For  example,  if  the  name  or  the  appearance  of  an  object 
is  changed,  then  all  domain  elements  using  the  object  are  modified  to  reflect  the  change. 
Changes  to  a  relation  also  invoke  the  update  of  operators  and  states  that  use  the  relation. 
This  provides  flexibility  in  domain  development,  as  described  earlier. 

The  ability  to  define  operators  using  objects  high  in  the  hierarchy  allows  for  a  more  general 
operator  definition,  thereby  consolidating  information  and  making  the  domain  better 
organized.  This  is  demonstrated  in  the  cooking  carrot  cake  domain  in  Appendix  C.  The 
operators  get  and  put-in-bowl  are  defined  using  the  super  class  object  ingr.  This  means  that 
separate  definitions  are  not  needed  for  carrots,  spices,  sugar,  oil,  and  flour.  This  makes  the 
domain  easier  to  understand  and  maintain 

As  operators  or  states  are  being  developed,  sometimes  the  expert  will  describe  object 
connections  that  have  not  yet  been  defined  as  a  relation.  When  the  system  detects  an 
undefined  connection,  a  warning  message  is  produced,  giving  the  expert  an  opportunity  to 
add  the  needed  relation  knowledge.  An  expert  would  create  an  undefined  connection 
between  objects  because  during  operator  development  the  expert  has  visual  cues  of  the 
needed  information  for  the  operator.  Thus  when  the  expert  thinks  of  a  needed  relation  the 
connection  is  made.  The  system  then  uses  this  new  information  to  solicit  the  missing 
relation  definition  from  the  expert  For  example,  in  the  machining  domain,  a  relation 
between  the  drill  machine  and  the  drill-bit  may  not  have  been  initially  defined.  During  the 
building  of  the  drill  operator,  the  drill  machine  and  drill-bit  are  connected  together.  It  is 
visually  obvious  to  have  the  drill  holding  a  drill-bit  in  order  to  drill  a  hole  in  a  part  The 
system  recognizes  that  no  relation  definition  exists  between  the  drill  machine  and  the  drill- 
bit  and  warns  the  user  of  this.  The  user  can  then  define  the  holding-tool  relation  between 
the  drill  machine  and  the  drill-bit 

The  final  issue  that  aids  in  the  development  of  large  domains  is  to  provide  the  expert  a 
direct  manipulation  interface  for  organizing  the  workspace.  Because  the  interface  is  easily 
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modifiable,  die  expert  can  organize  the  graphical  objects  according  to  personal  preference 
and  for  productive  development  Figure  5. 1  depicts  the  bottom  half  of  the  operator  window 
with  the  medium  size  machining  domain  loaded.  Notice  that  I  have  grouped  prototype 
objects  together  by  types  (e.g.,  row  of  drill  bits,  row  of  machines,  etc.).  Also,  the  operators 
are  group  by  functionality  {e.g.,  operators  to  do  drilling  are  in  the  left  comer,  operators  to 
mill  parts  are  grouped  together,  etc.).  This  ability  to  manipulate  the  workspace  helped  to 
create  an  organized  environment  for  rapid  domain  development 
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Figure  5JL:  Operator  Window:  machining  domain  organized  at  bottom  of  window. 
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Chapter  6  -  Conclusion 


This  chapter  highlights  the  technical  accomplishments  of  this  dissertation  and  illuminates 
the  possibilities  far  future  research. 

6.1  Summary  of  Findings 

The  ability  to  model  real  world  information  with  metaphors  similar  to  the  visual  aspects  of 
the  domain  has  been  shown  to  aid  domain  development  This  is  because: 

•  Objects  are  based  on  the  physical  world.  The  objects  in  APPRENTICE  occupy 
space  on  the  screen  similar  to  the  space  that  objects  occupy  in  the  real  world.  Also 
similar  to  the  physical  world  APPRENTICE  objects  are  only  at  a  single  location  at 
a  time.  These  similarities  help  the  expert  understand  and  describe  a  domain  using 
already  developed  intuitions. 

•  In  using  already  learned  intuitions  to  develop  domains  in  APPRENTICE  the 
experts  do  not  have  to  mentally  translate  the  physical  domain  into  a  foreign 
machine  representation. 

•  When  the  system  is  developing  a  solution  to  a  problem,  the  expert  can  quickly 
understand  what  tire  system  is  doing  by  observing  tire  animation.  This  allows  the 
expert  confidence  in  the  system  and  provides  a  faster  indicator  when  some 
knowledge  is  incomplete. 

6.2  Contributions  of  This  Research 

In  this  research,  a  method  for  graphical  knowledge  acquisition  for  visual  planning  domains 
was  developed,  described,  implemented,  and  tested.  A  strong  visual  intuition  for  physical 
and  conceptual  domains  maps  very  well  with  the  conceptual  representation  of  the  domain. 
With  these  methods,  the  expert  is  able  to  express  domain  information  similar  to  the  way 
the  information  is  thought  about  These  techniques  were  shown  to  increase  the  ease,  speed, 
and  accuracy  of  knowledge  acquisition  through  a  set  of  user  studies. 
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The  APPRENTICE  system  is  tightly  integrated  into  the  Prodigy  planning  system. 
Although  APPRENTICE  is  easy  to  use,  it  does  not  keep  the  expert  from  using  the 
expressive  power  of  the  underlying  planning  system  if  needed.  APPRENTICE  allows  the 
building  of  a  Hnwniin  using  a  graphical  representation  to  create  the  information.  Domain 
elements  such  as  objects,  relations,  and  operators  are  defined  graphically.  This  allows  the 
expert  a  straightforward  mapping  between  the  physical  domain  and  the  encoded 
representation. 

Several  studies  were  done  with  the  APPRENTICE  system  to  evaluate  the  ease,  speed,  and 
accuracy  of  the  new  techniques.  Study  1  had  32  subjects  developing  their  individual 
domains  in  APPRENTICE.  This  study  showed  the  ease  of  use  and  the  flexibility  of  the 
system  with  multiple  subjects.  Study  2  was  a  comparison  between  the  productivity  of 
developing  domains  in  a  text  editor  versus  developing  domains  in  APPRENTICE,  using 
different  types  of  users.  The  APPRENTICE  system  produced  quicker  development  time 
and  better  domain  understanding  for  all  but  the  most  seasoned  Prodigy  users.  In  Study  3, 
die  system  was  used  by  a  subject  to  develop  four  similar  domains  over  time.  This  study 
indicates  that  domain  development  efficiency  increases  over  time  as  the  subject  becomes 
more  familiar  with  the  system.  Finally,  in  Study  4  the  system  was  used  to  develop  a 
machining  domain.  This  study  verified  the  ability  of  APPRENTICE  to  woik  with  a  larger 

rtrtnrmin 

An  important  contribution  is  that  APPRENTICE  demonstrated  die  soundness  of  the 
techniques  even  with  users  with  very  little  prior  computer  knowledge.  In  Study  2  the  non¬ 
technical  computer  users  were  able  to  develop  working  domains  using  the  APPRENTICE 
system,  but  most  of  them  were  not  able  to  develop  similar  domains  using  Emacs.  Even 
with  the  non-tec hnical  subject  who  was  able  to  build  a  domain  in  both  systems,  there  was 
still  about  a  300%  improvement  in  the  development  time  for  the  APPRENTICE  system. 

63  Future  Worfc 

As  I  developed  the  core  techniques  for  APPRENTICE,  more  ideas  emerged  than  I  had 
time  to  investigate.  Some  of  these  ideas  would  help  make  the  system  more  usable,  while 
others  could  be  used  to  extend  the  system  capabilities.  I  will  describe  the  ideas  in  the 
following  sections. 


63.1  APPRENTICE-assisted  Search  Control  Rule  Development 
la  this  dissertation  I  have  only  discussed  the  creation  of  factual  knowledge  for  a  domain. 
Search  control  knowledge  is  also  important  in  encoding  information  to  better  guide  the 
planning  system.  Future  work  should  be  done  to  aid  die  expert  in  developing  these  search 
control  rules.  This  section  outlines  possible  techniques  that  could  be  incorporated  into  the 
system. 

The  goal  of  a  planning  system  is  to  find  a  set  of  operators  that,  when  applied  sequentially, 
will  transform  the  initial  state  into  the  goal  state.  The  search  sequence  is  determined  by  the 
accuracy  of  the  operators,  the  correct  specification  of  the  initial  and  goal  states,  and  the  set 
of  defined  search  control  rules.  Thus,  the  efficiency  of  the  planner  is  dependent  on  the 
selection  of  goals,  operators,  and  objects  in  exploring  the  problem  space.  This  selection  can 
be  directly  controlled  by  search  control  rules.  Search  control  rules  can  be  developed  by  both 
the  user  and  system,  by  directly  observing  and  correcting  search  path  mistakes. 

There  are  three  phases  of  development  in  creating  search  control  rules.  These  phases  are:  1) 
determining  if  the  system  is  solving  the  problem  correctly;  2)  if  the  system  is  not  solving 
the  problem  correctly,  determining  why  and  where  the  knowledge  base  is  incomplete;  and 
3)  creating  search  control  rules  to  enhance  the  knowledge  bast. 

To  automate  the  development  process  the  expert  needs  to  determine  where  the  planner 
made  a  wrong  move  and  to  show  the  planner  die  correct  move  that  should  have  been 
made.  The  system  could  ask  directed  questions  that  would  allow  it  to  automatically 
formulate  a  control  rule  for  current  and  future  use. 

To  demonstrate  a  possible  scenario  1  will  use  a  STRIPS  work!  type  domain.  The  domain  is 
as  follows: 

object:  package,  robot,  and  rooms 

relation:  in- room- package,  package -on -robot ,  in-room- robot,  rooms - 
connected 

Operator:  pickup-package,  put  -down-package ,  move -robot 

initial  State:  ( in-rocm-package  pkl  RoomA)  ,  ( in-room-robot  Robot  1  RoomA) , 

( room-connected  RoomA  RoomB,  (room-connected  RoomB  RoomA) 
goal  State:  ( in-room- robot  Robotl  P.oomB) ,  ( in-room-package  pkl  RoomB) 
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Note:  pickup-packag*  removes  the  package  from  die  room  that  the  package  is  in,  and 
put-down-packag*  places  the  package  in  the  same  room  as  the  robot 


Figure  6.1:  Example  of  a  STRIPS  world  type  domain. 

When  running  the  above  problem  the  expert  sees  the  robot  move  to  RoomB,  then  come 
back  to  pick  up  pkl,  and  then  move  to  RoomB  with  the  package.  This  is  because  the  first 
goal  is  to  get  ( in-roam-robot  Robot  1  RoomB) .  By  watching  this  solution  the  expert 
notices  that  die  robot  should  have  picked  up  the  pkl  first,  then  tried  to  move  to  die  next 
room. 

The  expert  then  demonstrates  to  the  system  that  die  robot  should  pick  up  the  package  first 
before  moving  the  robot  The  system  then  notes  that  there  is  a  discrepancy  between  what  it 
did  versus  what  it  should  have  done.  The  system  then  sets  about  obtaining  additional 
information  it  needs  to  avoid  similar  mistakes  like  this  in  the  future. 

Control  rules  take  the  form  of  selecting,  deleting,  or  preferring  some  information.  Much 
could  be  done  to  help  the  expert  write  these  rules.  The  important  information  to  determine 
is  what  kind  of  search  control  rule  needs  to  be  written  and  what  state  information  is  needed 
in  order  to  use  the  control  rule.  The  system  could  ask  focus  questions  of  the  expert  These 
questions  help  develop  control  rules  such  as  the  following:  if  die  goal  of  getting  a  package 
to  a  room  has  not  yet  been  achieved  and  the  robot  is  in  the  same  room  as  the  package,  then 
prefer  working  on  the  in-room-package  goal  first  This  control  rule  could  be  written  as 
follows. 
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( RRORDRR -CARD  X DATE - GOAL - RULE - 1 

(lhs  (and  (currant -noda  <n») 

( candidate -goal  <n>  ( in-roon  <Robby>  <room>) ) 
(candidata-goal  <n>  ( in-rooa-package  <pkl>  <room>) ) 

(known  <n>  (and  ( in ~roo»- package  <pkl>  <RoonA>) 

(not -equal  <room>  <RoomA>) ) ) ) ) 

(rha  (prefer  goal  ( in- room-package  <pkl>  <room>) 

(in-roon  <Robby>  <room>) ) ) ) 

Figure  62:  Example  of  possible  control  rule  built  with  system  assistance. 

The  above  example  has  outlined  a  technique  for  possibly  adding  a  search  control  rale  to  a 
knowledge  base.  This  has  been  achieved  by  first  noting  that  the  planned  solution  was  not 
optimal,  next  demonstrating  to  the  system  the  correct  sequence,  and  finally  getting  system- 
directed  help  to  determine  the  relevant  information  that  was  needed  to  build  a  new  search 
control  rule.  A  deeper  understanding  of  how  a  system  can  interactively  aid  an  expert  in 
developing  search  control  rules  will  improve  domain  development 

632  Seamless  Environment:  Visual  and  Textual  Representation 
One  of  the  things  that  Study  2  revealed  was  that  Prodigy  experts  built  domains  faster  in 
Emacs  than  in  APPRENTICE.  This  suggests  that  the  ability  to  build  domains  in  Emacs 
should  be  incorporated  into  the  APPRENTICE  system.  Providing  a  seamless  bridge 
between  the  power  of  the  graphical  interface  along  with  the  ability  to  use  Emacs  to  build 
domains  could  prove  to  be  a  dynamic  combination. 

Currently  APPRENTICE  supports  full  data  flow  from  the  graphical  system  to  Emacs. 
APPRENTICE  creates  the  Prodigy  code  automatically  from  the  graphical  description  of 
the  domain.  This  created  code  can  be  edited  in  Emacs.  However,  the  Emacs-edited  code 
does  not  automatically  invoke  a  graphical  representation  in  the  graphical  system.  Thus,  the 
graphical  interface  cannot  be  used  to  edit  the  Emacs  code.  Figure  6.3  shows  this 
relationship. 


Figure  63:  Currently  APPRENTICE  provides  full  data  flow  from  the  graphical  interface 
to  Emacs,  but  only  limited  data  flow  from  Emacs  to  the  graphical  interface. 
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Further  research  could  explore  having  text  that  corresponds  to  graphical  relations  developed 
with  a  text  editor  automatically  update  the  graphical  display.  This  would  allow  editing  of 
the  textual  code  with  the  graphical  system. [Glinert  90]. 

633  Spatial  Multidimensional  Relations 

Daring  the  course  of  exploring  domains  in  APPRENTICE,  it  became  apparent  that  for 
development  of  some  domains  two-  and  three-dimensional  spatial  information  needs  to  be 
easily  represented.  This  would  be  needed  in  a  CAD/CAM  domain,  for  example.  In  these 
types  of  domains,  information  is  thought  of  as  spatial  positioning  between  objects.  More 
work  needs  to  be  done  into  how  to  allow  the  expert  an  easy  and  intuitive  way  to  represent 
and  define  this  spatial  positioning  [Chang  90]. 

63.4  Apprentice  Techniques  for  Non-visual  Domains 
The  APPRENTICE  techniques  have  proven  to  enhance  the  domain  development  process 
for  visual  planning  domains.  This  is  partially  due  to  the  ability  of  experts  to  understand  the 
domain  development  process  in  respect  to  how  they  describe  the  domain.  It  may  be 
possible  to  develop  techniques  similar  to  the  ones  used  in  APPRENTICE  for  non-visual 
domains  that  will  also  help  enhance  the  development  process. 
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Appendix  A  -  Results  Chart  from  Study  1 


Table  A.1:  Study  1  Subjects 


Chart  1  is  used  in  conjunction  with' Table  1  to  help  decipher  the  student  description  column. 
This  information  is  broken  down  by  each  student's  year,  department,  and  major. 


Department 

Major 

CFA  -  College  of  Fine  Aits 

BSC  -  Biological  Sciences 

CMU  -  Carnegie  Mellon  University 

HOO  -  General  HSS 

HSS  -  Humanities  and  Social  Science 

IA  -  Industrial  Management 

IM  -  Industrial  Management 

INI  -  Information  Networking  Institute 

MCS  -  Mellon  College  erf  Science 

IMC  -  JM-CIT/MCS  Track 

SCS  •  School  of  Computer  Science 

PHI  •  Philosophy 

SIA  -  Graduate  School  of  Industrial 

PSY  -  Psychology 

Administration 

MTH- Math 

CMU  -  Carnegie  Mellon  University 

MUS  -  Music 

HSS  •  Humanities  and  Social  Science 

M-C  -  Math  and  Computer  Science 

IM  •  Industrial  Management 

RI- Robotics 

MCS  -  Mellon  College  of  Science 

SDS  -  Social  and  Decision  Science 

SCS  •  School  of  Computer  Science 

SIA  •  Graduate  School  of  Industrial 

Administration 

Table  AJ:  Student  Categories 
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Appendix  C  -  Selected  Domains  from 

Study  1 


Subject  1 
Subject  12 
Subject  22 
Subject  24 
Subject  29 


Customer  Ordering  Domain  Built  By  Subject  1 


Automatically  Generated 
ftodigy  Domain  Cock  far  Subject  1 

(ia-a  ocdK  eypa) 

IlM*  qw 
tia-a  Idncban  cypat 
li«  morn—  cypal 
lia-a  ntoc  cypal 


*>>> 


Start  and  Goal  StatemAPHUZNTICE  for  Subject  1 
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Automatically  Generated  Start  AGoai  State  Description  for  Subject  1 
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Start  and  Goal  Stale  in  APPRENTICE  fix  Subject  12 


Autenattafly  GenaaiBdStait&Goal  State  Description  fir  Sulject  12 
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Drip  to  Mecca  By  Subject  22 
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Operator  (corn) 


Autonrcaically  Generated  Start&Goal  State  Description  for  Subject  24 


Start  and  Goal  State  in  APPRENTICE  for  Subject  24 
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Cooking  Carrot  Cake  By  Subject  29 


Start  and  Goal  Stale  in  APfRENTKE  ffar  Subject  29 


taonatica&y  Geneaaed  Start  &  God  State  Description  &r  Subject  29 


>mvmui 


toil  aUMUt 


Automatically  Geonaied  ftodigy  Domain  Code 
for  Subject  29 
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Appendix  D  -  Information  Given  to 
Subjects  in  Study  2  Phase  1 


Prodigy  and  Domain  Description 

In  the  field  of  Artificial  Intelligence  one  way  of  emulating  human  problem  solving 
techniques  is  with  planning  systems.  Planning  systems  allow  a  user  to  define  a  situation 
and  problem  systematically  and  have  the  system  plan  a  solution.  PRODIGY  is  such  a 
general  purpose  planning  system. 

In  PRODIGY  the  actions  that  the  planner  takes  are  defined  by  operators  (actions  a  person 
would  use  to  solve  a  problem).  The  problem  is  a  coarse  representation  of  relevant  facts 
describing  the  initial  situation  before  apian  is  executed;  and  tire  final  situation  desired  after 
a  set  of  operators  are  executed.  Let's  take  a  very  simple  example  like  the  process  of 
delivering  a  pizza.  This  can  be  a  very  complicated  process,  but  in  this  case  we  only  want  a 
very  coarse  grain  model  of  the  "domain"  or  situation. 

Domain:  You  have  a  car  and  work  for  a  pizza  place.  When  the  pizza  is  ready  you  load  it 
into  your  car,  which  is  at  the  pizza  place,  and  deliver  it  to  someone's  home.  For  this  domain 
you  can  only  deliver  one  pizza  at  a  time.  There  are  3  operators  that  you  will  be  concerned 
with:  load-pizza-in-car,  drive-to-location  and  unload-pi  zza-from-car.  The  load-pizza-in-car 
states  that  if  the  pizza  is  ready  and  the  car  is  at  the  same  location  as  the  pizza  and  the  car  is 
empty  then  the  pizza  can  be  put  inside  the  car.  The  next  operator,  drive-to-location,  states 
that  if  a  car  is  in  at  locationl  then  it  can  drive  to  a  different  location,  location2.  Finally  the 
unload-pizza-froro-car  stales  that  if  a  pizza  is  in  the  car  then  the  pizza  can  be  unloaded  from 
the  car  and  the  pizza  is  now  in  the  same  location  that  the  car  is  and  the  car  is  empty. 


load-pizza-in-car 

drive-car 

unload-pizza- from-car 

if 

if 

if 

pizza  at  locationl 

car  at  locationl 

car  at  location 

car  at  locationl 

that 

pizza  in  car 

then 

caratlocation2 

then 

unload  pizza  to  location 

Lets  describe  two  problems: 

PROBLEM  1:  The  initial  situation  is  that  the  pizza  and  the  car  are  at  die  store.  The  goal  is 
to  deliver  the  pizza  to  Larry's  house.  The  solution  is  to  load-pizza-in-car  at  the  store,  drive- 
car  to  Larry's  house  and  then  unload  the  pizza  at  Larry's  house. 

PROBLEM  2:  Here  is  a  slightly  more  complicated  situation.  The  initial  situation  is  that  the 
car  is  now  at  Larry's  house  and  the  pizza  is  at  the  pizza  place.  Again  the  goal  is  to  deliver 
die  pizza  to  Larry's  house.  Here  the  solution  is  to  drive-car  to  the  store  then  load-pizza-in¬ 
car  at  the  store,  drive-car  to  Larry's  house  and  then  unload  the  pizza  at  Larry's  house. 

How  do  you  encode  this  information  so  a  computer  can  use  it?  That  is  what  planning 
systems  allow  you  to  do. 

The  pizza  delivery  domain  is  right  now  a  very  simple  domain,  but  by  adding  more  delivery 
locations  and  route  length  information  the  complexity  of  the  problem  increases  and  the 
system  can  actually  be  useful  in  planning  pizza  delivery  routes. 
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The  task  is  to  build  this  simple  pizza  delivery  route  domain  as  an  exercise  in  Prodigy.  You 
will  use  Apprentice,  a  visual  domain  building  tool,  and  Emacs,  a  textual  editor. 


Emacs/Prodigy  Example  Domain 


L  Introductioo 

Writing  a  domain  in  Prodigy  requires  several  steps.  You  must  create  object  type  definitions 
used  in  the  domain,  operators,  instances  of  objects,  start  states,  and  goals  states.  Below  is 
an  example  of  all  these  steps  for  the  pizza  domain.  We  will  use  the  pizza  domain  to  define 
a  domain  in  Prodigy  using  Emacs. 

2.  Defining  Object  Types 

First  you  mm  define  object  definitions  for  a  domain.  These  type  definitions  tell  Prodigy 
the  name  of  the  objects  you  will  be  using  in  the  domain.  For  die  pizza  domain  you  will 
need  pizza  objects,  car  objects  and  location  objects.  An  object  definition  has  the  following 
skeleton  (IS-A  object-name  TYPE).  For  the  pizza  domain  these  objects  are  defined  as 
below. 


QS-A  Loqrica  ModriTYPB) 

(IS-A  Caraodal  TYPE) 

(IS-A  PbBModal  TYPE) 

3.  Defining  Predicates  and  Operators 

Next  you  must  define  operators.  The  operators  define  how  to  change  states  in  the  domain. 
This  change  is  defined  by  giving  the  preconditions  of  a  situation  and  then  die  effects  of  the 
change.  These  pre-conditions  and  effects  are  represented  by  predicates.  Predicates  represent 
relations  between  objects.  In  order  to  allow  an  operator  to  work  in  a  generic  way  the 
predicates  use  variables.  These  variables  match  different  objects  in  the  state  definition  and 
are  represented  by  o  surrounding  the  name  of  die  variable.  An  operator  skeleton  is  as 
shown  below: 


(OPERATOR  mamt  of-optrator 
(PARAMS 

((<wl>  typ«l) 

(v«2>  type?) 

(<*«a>  cyps3)  _) 

(PRBCONDS 

(AND  (pndl  <varl>  <y*i2>) 

_  (jpnd2<w2>  <nt3>))~) 

(EFFECTS 

((DEL  (pndl  <varl>  <m 2>» 
(ADD  (pnd3  <*arl>  <w3>)))  —  ) 


Real  operators  in  die  pizza  domain  are  defined  below.  Look  at  the  operator  LOAD-PIZZA. 
In  the  PARAMS  list  die  variable  <pizza>  is  of  type  pizza-model,  etc.  In  the  precond  list  the 
condition  Car-at-loc  says  that  the  variable  <car>  has  to  be  at  variable  location  <loc>  and  the 
condition  pizza-at-loc  says  that  the  variable  <pizza>  has  to  be  at  the  same  location  <loc>  as 
the  variable  <car>.  Finally  if  these  predicates  are  true  in  the  state,  then  the  variable  <pizza> 
is  moved  inside  the  <car>  variable  and  the  <pizza>  is  no  longer  at  the  location  <loc>.  See  if 
you  can  determine  what  the  other  operators  mean. 

(OPERATOR  LOAD-PEZA 
(PARAMS 

((«sm>  PIZZA-MODEL)  (<fa>o  LOCATION-MODEL)  (<cn>  CAR-MODEL))) 

(PRBCONDS 

(AND  (CAR-AT-LOC  <ca>  <too)  (PIZZA-AT-LOC  <pdu>  <kx>))) 

(EFFECTS 

((DEL  (PIZZA-AT-LOC  <piza>  <kx>)) 

(ADD  (PIZZA-IN-CAR  <puz»  <**»))» 
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(OPERATOR  MOVBCAR 
(PARAMS 

C(<fc*l>  LOCATION-MODEL)  (<k> iO>  LOCATION-MODEL)  (<cae>  CAR-MODEL))) 

(PRBCONDS  (CAR-AT-LOC  <at>  <docl>)) 

(HFFBCTS 

((DEL  (CAR-AT-LOC  <tm  >  <locl>)) 

(ADO  (CAR-AT-LOC  <c*e>  Ooe2>))))) 

(OPERATOR  UNLOAD-PEZA 
(PARAMS 

((<loo  LOCATION-MODEL)  (<pizza>  PIZZA-MODEL)  (<cw>  CAR-MODEL))) 

(PRBCONDS 

(AND  (PIZZA-IN-CAR  <pis»  <ca>)  (CAR-AT-LOC  <eu>  <doo))) 

(BPFBCTS 

((PEL  (PIZZA-IN-CAR  <pia»  <eu>)) 

(ADO  (PIZZA-AT-LOC  <pm>  <Io o))») 

4.  Defining  a  State 

After  the  Hntmrin  is  described  then  the  problems  that  you  want  to  solve  need  to  be  defined. 
The  problem  is  defined  by  a  start  state  (STATE,  below)  and  a  gaol  state  (GOAL,  below)  in 
the  domain  Below  the  State  says  that  the  Pepperoni  and  Domino-Car  are  at  the  Pizza-place 
and  the  goal  is  to  get  the  Pepperoni  to  Larry's  place.  When  defining  problems  for  a  domain 
you  must  specify  what  the  instances  are  in  the  problem.  For  example  the  word  Pepperoni  is 
an  arfnai  pizza  and  Domino-Car  is  an  actual  car. 

(HAS-INSTANCES  Localm-Modei  Piaa-PUoe) 

(HAS- INSTANCES  Locum-ModU  Liny) 

(HAS-INSTANCES  CaromW  Dommo-Cir) 

(HAS-INSTANCES  Pisa-amU  Peppered) 

(GOAL  (PEZA-AT-LOC  PEPPERONI  LARRY)) 

(STATE  (AND  (PIZZA-AT-LOC  PEPPERONI  PIZZA-PLACE)  (CAR-AT-LOC  DOMINO-CAR  PIZZA-PLACE))) 

5.  Emacs  and  Prodigy  Usage  Description 

You  will  have  three  windows  available.  The  first  window  will  be  the  domain  window,  the 
second  window  will  be  the  start  and  goal  window  and  the  third  window  will  be  the  lisp 
window. 

♦ 

Text  is  entered  in  each  window  by  typing.  The  text  is  placed  at  the  point  of  the  blinking 
cursor  I  in  the  active  window.  A  window  becomes  active  by  clicking  the  mouse  in  it  The 
cursor  can  be  moved  within  a  window  by  pointing  and  clicking  the  mouse  button  at  the 
desired  location  or  with  the  use  of  control  key  +  text  keys.  The  cursor  movements  for 
different  keys  are  listed  below.  A-<key>  means  hold  the  control  key  down  with  the 
respective  <key>. 

*-a  -  move  cursor  to  the  front  of  the  line 
A-f  -  move  cursor  forward  one  space 
A-n  -  move  cursor  down  one  line 
A-g  -  abort  whatever  you  did 

6.  Starting  Prodigy 

Once  you  have  created  the  objects  and  die  operators  for  a  domain  then  a-cM  will  load  that 
information.  When  you  have  typed  in  a  problem,  A-cAp  wifi  load  the  problem  and  run 
Ptodigy.  The  results  of  the  run  will  be  printed  out  in  the  lisp  window.  You  can  also  test 
each  operator  separately  against  the  current  system  state.  To  start  testing  individual 


A-e  -  move  cursor  to  the  end  of  the  line 
A-b  -  move  cursor  backward  one  space 
A-p  -  move  cursor  up  one  line 
ar-Aq  -  move  to  the  above  window 
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operators  use  the  *-0-^  command.  Then  use  the  step-operator  function  in  the  lisp  window 
to  run  operators.  Below  is  the  syntax  for  the  step-operator: 

(step-operator  <op-name>  (<varl>  VAR1-NAME)  (<var2> .  VAR2-NAME)) 

Example  of  step-operator  from  pizza  domain  above: 

(step-operator  load-pkza  (<pizza>  .  Pepperoni)  (<car>  .  Domino-Car)  (<loc>  Pizza- 
Place)) 

To  view  the  current  state  type:  (print-state) 

7.  Steps  for  Defining  the  Pizza  Domain  and  Problem 

1)  Define  the  object  types  (car,  pizza,  and  location) 

2)  Define  needed  relations  (pizza-in-car,  car-at-loc,  and  pizza- at-loc) 

3)  Define  the  operators  (load-pizza,  unload-pizza,  and  chive-car) 

4)  Define  the  problem  with  instances  (Pepperoni,  Domino-car,  Pizza-Place,  and  Larry- 
Place) 

3)  Define  the  start  state  and  the  goal  state 

6)  Debug  domain  (fix  syntax  errors  and  step  through  individual  operators) 


APPRENTICE  Description 


L  Introduction 

APPRENTICE  allows  an  expert  to  create  each  part  of  a  domain  in  Prodigy  using  pictures 
that  the  expert  develops.  There  is  an  APPRENTICE  development  window  for  each  aspect 
of  a  domain.  To  make  domain  development  easier,  the  main  input  device  is  a  mouse.  In 
each  window  there  are  buttons  that  perform  actions  and  textual  items  that  display 
information.  Each  window  has  a  development  box  that  the  expert  works  in  to  develop  a 
particular  piece  of  knowledge. 

Users  familiar  with  the  Macintosh  interface  should  be  very  comfortable  using 
APPRENTICE.  Every  item  other  than  the  development  box  is  movable  by  moving  the 
mouse  to  a  position  over  the  object  and  holding  the  left  mouse  button  down  while  moving 
the  object  to  die  desired  location. 

2.  APPRENTICE  Windows 

Model  Window  -  Allows  objects  in  a  domain  to  be  defined  (e.g.,  Track,  Pizza,  Location) 
Relation  Window  -  Allows  relationships  between  objects  to  be  defined  (e.g.,  inside-track, 
at-location). 

Operator  Window  -  Allows  state  changes  to  be  defined  (e.g.,  load-pizza,  move-car,  unload- 
pizza). 

State  Window  -  Allows  the  definition  of  a  start  state  and  a  goal  state. 

Problem  Window  -  Allows  the  setup  and  running  of  a  problem  to  be  solved. 

Common  Apprentice  Window  Facts 


•  Halid  down  button  over  an  object  move  it 
'  Hold  down  button  in  blank  oca  and 

rectangle  select  object  inside  rectangle. 

•  Shift  click  batten  select  multiple  object. 


-  ,  .  •  dick  on  object  ailowi  editing  information. 

dxag— — Pm  M  []K  ’  die*  retarn  in  watt  lrindow  ailowi  work  on  mother  thing. 

.  Shift  dick  on  anything  give  help  menage  about  thing  dieted. 
♦  All  Click  on  object  ailowi  editing  attributes  of  the  object 


•  Selected  items  appear  highlighted. 

•  Shift  right  click  on  an  item  gives  a  help  message  on  the  item. 

•  The  name  text  represents  the  name  of  a  item  (i.e.,  object,  relation,  operator,  state  or 

problem). 

•  Multiple  items  selection:  1)  hold  the  shift  key  down;  2)  drag  a  box  around  a  set  of  items. 

•  Text  can  be  edited  by  clicking  the  right  button. 

•  Object  names  can  be  changed  by  right  clicking  on  die  object. 

•  Buttons  can  be  moved  with  the  mouse  by  there  text  part  are  activated  by  the  box  part 

2.L  Model  Window 

The  model  window  allows  the  expert  to  develop  the  objects  of  a  domain.  The  objects  have 
an  appearance  as  well  as  connection  spots.  The  development  box  allows  you  to  work  on  a 
particular  object  Window  components: 


NAME:  name  of  the  particular  object  type 

GRAPHICAL  EDITOR:  An  area  used  to  build  the  visual  representation  of  objects 
DELETE  INST  BUTTON  -  Deletes  selected  model  instances 
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2.1.1.  Graphical  Editor 


Models  or  objects  are  made  up  of  simple  drawing  elements.  The  editor  lets  you  draw  these 
simple  elements  to  build  an  object  Each  element  in  the  editor  can  be  manipulated  with 
mouse  movement 

The  following  drawing  objects  are  available: 

[K1  Arrow  allows  selection,  movement  and  alteration  of  elements. 

f\1  Allows  you  to  draw  a  line  element 

[T~l  Allows  you  to  add  a  text  element 

fol  Allows  you  to  draw  a  box  element 

(SSI  Allows  the  instance  name  of  die  object  to  be  displayed. 

[ml  DEL  deletes  the  selected  elements. 

[cm]  Saves  die  elements  in  the  editor  to  the  actual  object  If  no  object  is  currently  being 
worked  on,  then  a  new  object  is  created. 

R  Allows  you  to  place  connection  point  element  on  the  object 

12.  Relation  Window 

Drag  the  objects  into  the  work  area  that  defines  a  relationship  between  die  objects.  Connect 
the  objects  to  one  another  with  their  connection  points  by  clicking  on  each  relevant 
connection  point  When  you  are  finished  with  a  relation  then  light  click  on  done. 

13.  Operator  State 

Operators  are  defined  by  dragging  objects  and  connecting  them  together  in  the  pre  state  and 
then  in  post  state.  The  Copy  prestate  button  copies  the  information  in  the  pre  state  side 
into  the  post  state  side.  The  Link  Obj  buttons  allows  an  object  selected  in  the  pre  state  to 
be  linked  with  an  object  selected  in  the  post  state.  When  you  are  finished  with  an  operator 
then  right  click  on  DONE. 

14.  State  Window 

To  define  a  state,  drag  the  appropriate  objects  into  the  work  area  window  and  connect  the 
objects  up  as  they  would  be  in  a  state.  Copy  state  button  will  copy  a  selected  state 
into  the  work  area.  When  you  are  finished  with  a  state  that  right  click  on  DONE. 

25.  Problem  Window 

The  problem  window  allows  you  to  define  a  problem  that  consists  of  a  start  state,  goal 
state,  and  a  set  of  operators.  The  problem  elements  are  defined  by  right  clicking  on  the 
START-STATE ,  GOAL-STATE,  and  OPS  text  Once  a  problem  is  defined,  to  have  the 
system  try  and  plan  a  path  click  the  RUN  DOMAIN  button.  If  you  want  to  test  on  operator 
at  a  time  click  the  STEP  FORWARD  button. 


3.  Steps  for  Defining  the  Pizza  Domain  and  Problem 

1)  Draw  the  object  types  with  connection  (car-model,  pizza-model,  and  location-model) 

2)  Define  needed  relations  (pizza-in-car,  car-at-loc,  and  pizza-at-loc) 

3)  Define  the  operators  by  pre  and  post  state  (load-pizza,  unload-pizza,  and  dnve-car) 

4)  Define  the  start  state  and  the  goal  state  graphically.  The  instances  should  have  the  names. 
Pepucroni,  Domino-car,  Pizza-Place,  and  Laity-Place 

5)  Define  the  problem  by  combining  operator,  start  state,  and  goal  state 

6)  Debug  domain  (fix  syntax  errors  and  step  through  individual  operators) 


Strips  World  Description 

Problem  Description 


The  strips  domain  is  a  fictitious  world  comprised  of  the  following  objects:  robots,  boxes, 
and  rooms.  Relations  between  the  objects  are  as  follows:  1)  robots  can  hold  a  box;  2)  two 
rooms  can  be  connected  together,  3)  robots  can  be  in  a  room;  and  4)  boxes  can  be  in  a 
room.  The  actions  that  are  performed  in  this  domain  are:  1)  robots  can  pick  up  boxes;  2) 
robots  can  put  down  boxes;  3)  robots  can  move  between  connected  rooms  with  a  box;  and 
4)  robots  can  move  between  connected  rooms  without  a  box.  The  goal  of  this  domain  is  to 
plan  paths  of  movement  for  boxes  to  get  from  an  initial  location  to  a  final  destination. 

Task:  Define  an  initial  state,  goal  state,  and  domain.  Have  the  system  find  the  plan  needed 
to  go  from  initial  to  goal  state  of  the  following  problems. 

Problem  1:  For  the  initial  state  create  two  rooms  (Rooml,  Room2),  a  robot,  and  a  box. 
Rooml  and  Room2  are  connected  together.  Have  the  robot  and  box  in  Rooml.  The  goal  is 
to  have  the  box  placed  in  Room2. 

Problem  2:  Use  the  work  from  the  previous  problem.  For  the  initial  state  create  three 
rooms  (Rooml,  Room2,  Room3),  a  robot,  and  a  box.  Rooml  is  connected  to  Room2  and 
Room2  is  connected  to  Room3.  Have  the  robot  in  Rooml  and  the  box  in  Room2.  This 
time  the  goal  state  is  to  have  the  box  be  put  in  Room3. 
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Logistic  World  Description 
Problem  Description 


The  logistic  world  is  comprised  of  the  following  objects:  boxes  and  hubs.  Hubs  are 
buildings  to  which  boxes  are  brought  for  sorting  and  distribution.  Relations  between 
objects  are  as  follows:  1)  boxes  can  be  at  the  indoor  of  a  bub  2)  boxes  can  be  at  the  outdoor 
of  a  hub  and  3)  hubs  are  connected  together  from  the  outdoor  of  one  hub  to  the  indoor  of 
another  hub.  The  actions  in  the  domain  are  that  1)  boxes  can  move  between  the  outdoor  of 
one  hub  to  the  indoor  of  a  connected  hub  and  2)  a  box  at  a  hub's  indoor  must  be  sorted  to 
get  to  the  hub's  outdoor.  The  goal  of  this  domain  is  to  plan  paths  of  movement  for  boxes  to 
get  from  an  initial  location  to  a  final  destination. 

Task:  Define  an  initial  state,  goal  state,  and  domain.  Have  the  system  find  the  plan  needed 
to  go  from  initial  to  goal  state  of  the  following  problems. 

Problem  1:  The  initial  state  has  three  hubs  (hubl,  hub2,  and  hub3).  Hubl's  outdoor  is 
connected  to  Hub2's  indoor.  Hub2's  outdoor  is  connected  to  Hub3's  indoor.  There  is 
initially  a  box  (Boxl)  at  the  inside  door  of  Hubl.  The  goal  is  to  have  the  box  delivered  to 
the  inside  door  of  Hub3. 

Problem  2:  Use  the  work  from  the  previous  problem.  In  this  problem  the  initial  state  has 
four  hubs  (hubl,  hub2,  hub3  and  hub4).  Hubl's  outdoor  is  connected  to  Hub2's  and 
Hub3's  indoor.  Hub3's  outdoor,  is  connected  to  Hub4‘s  indoor.  A  box  (Boxl)  starts  at  the 
inside  door  of  Hubl.  The  goal  is  to  have  the  box  delivered  to  the  inside  door  of  Hub4. 
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Appendix  E  -  Questions  from  Study  2 

Phase  2 


E4 


Questions  from  Study  2  of  Phase  2 

The  lbflowing  section  contains  the  rintnam*.  questions,  and  results  from  study  2  of  phase  2.  For  this  phase  two 
rinmaim  were  used:  the  monkey  and  banana  dnmam  and  the  Modes  domain.  The  questions  are  about  operators  or 
states  in  these  domama. 

RaHi  question  section  has  a  graphical  representation  of  an  operator  or  state  along  side  the  textual  representa¬ 
tion  of  the  ««n*  information.  The  textual  representation  was  automatically  generated  with  APPRENTICE  from  the 
grephir*!  representation.  The  multiple-choice  question  is  below  these  representations.  The  correct  answer  to  the 
question  has  been  made  bold  for  your  convenience.  To  the  right  of  the  question  is  the  data  showing  the  subjects 
that  answered  this  question  incorrectly.  Only  one  incorrect  answer  was  given  during  the  graphical  representation 
taring  Hie  incorrect  question  for  the  textual  representation  will  be  duly  noted. 

Blocks  Domain 


Question  1: 

n— -  n*m~m  [~~|iwa> 
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OPERATOR  NAME»p2 

(<bottotn>  block-model) 
(<top>  block- model) 

(<aan>  ann-model) 
pncoods 

(on  <rop>  <bodom>) 
(empty-arm  <an») 

(clear  <top?) 
effects 

(dal  (on  <top>  <  bottom?)) 
(dal  (empty-ann  <ann>» 

(dal  (dam  <top>)) 

(add  (bolding  <xop>  <ann>» 
(add  (dev  <bottotn>» 


If  op2  has  fired  is  the  arm  empty  or  holding  a  block? 

1)  Holding 

2)  Empty 

3)  Unknown 


AEBnpkgree S2t  I 
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Question  2: 
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OPERATOR  NAME;  op3 


(<ann>  arm-model ) 
(<blodo  block-model) 
(<tabie>  table-model) 


(oo-tabU  <biock?  cable?) 
(empty-ann  <ann>)  (clear  <block>) 


(dal  (oo-tabie  cbiock?  <tab*e>)) 
(dal  (empty-ann  <arm>)) 

(dal  (dav  <biock>» 

(add  (holding  <  block?  <arm>)) 


If  op3  has  fired  is  the  arm  empty  or  holding  a  block? 

1)  Holding 

2 )  Empty 

3)  unknown 


Question  3: 


OPERATOR  NAME:  opl 


(<tabta>  tabie-modd) 
(<block>  block-model) 
(<mn>  amt-model) 


(holding  <Mock>  <ann>) 
effccti 

(del  (holding  <bkx±>  <ann>)) 
(add  (on-table  <block>  <table>)) 
(add  (empty-arm  <azm>» 

(add  (dear  <block>)) 


Is  operator  opl  representing? 

1)  Picking  up  a  block  off  the  table 

2)  Putting  down  a  block  on  the  table 

3)  Stacking  a  block  on  another 

4)  Qnatacking  a  block  off  another 


AIEsputS2ti 
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Question  4: 


OPERATOR  NAME:op2 


Is  operator  op2  representing? 

1)  Picking  up  a  block  off  the  table 

2)  Putting  down  a  block  on  the  table 

3)  Stacking  a  block  on  another 

4)  Unstacking  a  block  off  another 


(<bottom>  block-model) 
(<top»  block-model) 
(<arm>  arm-model) 


(on  <top>  <bottom>) 
(empty-arm  <axm>) 
(cle»<top>) 


(del  (on  <top>  <bottom>)) 
(del  (empty-arm  <ann>)) 

(del  (clear  <top>)) 

(add  (holding  <rop>  <ara»)) 
(add  (dee  <bottom>)) 


Question  5: 
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OPERATOR  NAME;  op3 

(<arm>  KTtHoodel) 

(<biock>  Mock-model) 

(<tabie>  table-model) 
precoode 

(oo-ubie  <block>  <ttbie>) 
(empty-arm  <m>)  (dear  <biock>) 

(dal  (oo-tabla  <block>  <tafaie>)) 
(dal  (empty-ann  <ann>» 

(dal  (dear  <biock>» 

(add  (bolding  <biock>  <arm>)) 


Is  operator  op3  representing? 

1)  Ricking  up  a  block  off  the  table 

2)  Putting  down  a  block  on  the  table 

3)  Stacking  a  block  on  another 

4)  Unstacking  a  block  off  another 


Question  6: 
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OPERATOR  NAME;  op4 


(<hnttnm>  Mode-model) 
(<top>  block-model) 
(<an»  arm-model) 


(bolding  <tap>  <arm>) 
(dear  <boaom>) 


(dal  (bolding  <top>  <*rm>)) 
(dal  (dear  <bottotn>)) 

(add  (on  <top>  <boaom>)) 
(add  (empty-arm  <ann>)) 
(add  (dear  <top>)) 


Is  operator  op4  representing? 

1)  Picking  up  a  block  off  the  table 

2)  Putting  down  a  block  on  the  table 

3)  Stacking  a  block  on  another 

4)  Unstacking  a  block  off  another 


Question  7: 

1  □ — 


START  STATE 


MW 

dear 
dear 
(empty*m  ann) 
(on-table  c  table) 
(on- table  b  table) 
(oD-tabie  a  table) 

GOAL  STATE 


In  the  goal  state  or  problem  stack-3 -block-1  what  is  blockB  on  top  of? 

1)  BlockC 

2)  Block*  r  „ 

3)  The  Table  l  ~  , 

4)  None  of  the  above  No«*T«cbS2:4 


Question  8: 

—  n. —  n. 


— 

Mm 

START  STATE 


(empty-ann  arm) 
(on-table  c  table) 
(on-table  b  table) 
(on-table  a  table) 

GOAL  STATE 


In  the  start  state  of  problem  stack- 3 -block- 1  what  is  blockB  on  top  of? 

1)  BlockC  ___________ 

2)  BlockA  ter"’""';'1'- 

3)  the  Table 

4)  None  of  the  above 


Question  9: 


START  STATE 
(d«rc) 

(dnrb) 

(dart) 

(empty-m  «nn) 
(an-table  c  table) 
(on-table  b  table) 
(on-table  a  table) 

GOAL  STATE 
(dear  a) 

(an  be) 

(anab) 


In  the  goal  state  of  problem  stack-3 -block-2  what  is  blockB  on  top  of? 

1)  BloekC 

2)  BlockA 

3)  The  Table 

4)  None  of  the  above 


N**TedtS2i3 


Question  10: 

This  domain  is  for? 

1)  Stacking  Block* 

2)  Moving  Packages 

3)  Getting  bananas 
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Monkey  and  Banana  Domain 


Question  1: 


OPERATOR  NAME;  op2 

puna 

(<book>  book-model) 

(<Jocaboo>  location-model) 

(<monkey>  monkey-model) 

(<block>  block-model) 

(<bmn»  banana-model))) 
ptecoadi 

(on-block  <monkey>  <biocfc>) 

(under-book  <locafion>  <hook>) 
(biock-at-tocatioo  <block>  <locatiaa>) 
(on-ceiling  <banena>  <hook>))) 
effect! 

(del  (on-ceiling  <banana>  <hook>)) 

(add  (holding-banana  <banana>  <monkey>)) 


or  holding  the  bananas? 


1)  Bolding  the  bananas 


2)  Empty 

3)  Unknown 


Question  2: 


n— - 


OPERATOR  NAME:  op3 

panma 

(<to-k>»  location- model) 

(<firom-loo  location-model) 

(<mookey>  monkey -model) 
precoods 

(connected  <to-loo  cfiom-loo) 

(connected  <from-loo  <to-loo) 
(monkey-at-location  <monkey>  <from-ioo))) 
effects 

(del  (monkey-at-  location  <monkey>  <from-loo)) 
(add  (monkey-at-location  <monkey>  <to-loo)) 


If  op3  has  fired  is  the  monkey  in  the  same  location? 

1)  Same  location 

2)  Different  location 

3)  Unknown 
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Question  3: 


d** 


OPERATOR  NAME:  opl 

(<block>  block-model) 

(<location>  location-model) 

(<monkey>  monkey-model) 
praconda 

(monkey-gt-location  <monkey>  <locabon>) 
(block-at-locadon  <bk>cio  <location>) 
effect! 

(del  (monkey-  at-location  <moakey>  <tocatjon>)) 
(add  (on-block  <monkey>  <block>)) 


Is  operator  opl  representing? 

1)  Grabbing  the  bananas 

2)  Monkey  moving  himself 

3)  Monkey  moving  the  Block 

4)  Hookey  getting  on  Block 


Question  4: 
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OPERATOR  NAME:  op2 
Parana 

(<hooio  book-model) 

(<location>  location-model) 

(<monkey>  monkey-model) 

(<biock>  block-model) 

(<banana>  banana-model))) 
pnconda 

(on-block  <monkey>  <Mock>) 

(under-hook  <Jocanon>  <hook>) 

(bkxk-at- location  <blodo  <locadon>) 
(on-ceiling  <banana>  <hook>))) 
effete 

(del  (on-ceiling  <banane>  <book>)) 

(add  (bolding -banana  <banana>  <monkey>)) 


Is  operator  op2  representing? 

1)  grabbing  the  bananas 

2)  Monkey  moving  himself 

3)  Monkey  moving  the  Block 

4)  Monkey  getting  on  Block 


E- 


Question  5 


|  |o— ■  i  Quasi 


OPERATOR  NAME:  op3 

ptrams 

(<U»4oo  kxatiot- model) 

(<£rom-loo  location-model) 

(<m«key>  monkey-model) 
precood* 

(connected  <to-loo  <fimm-loo) 

(connected  <frotn-loo  <to-loo) 

(mookey-u- location  <monkey>  <&om-ioo))) 

(del  (monkey-at- location  <monkey>  <from-loo)) 
(add  (monkey-at-location  <monkey>  <to-loo)) 


Is  operator  op3  representing? 

1)  Grabbing  the  bananas 

2)  Monkey  moving  himself 

3)  Monkey  moving  the  Block 

4)  Monkey  getting  on  Block 


Question  6: 


|  |o—  Qimoi 


Is  operator  op4  representing? 

1)  Grabbing  the  bananas 

2)  Monkey  moving  himself 

3)  Monkey  moving  the  Block 

4)  Monkey  getting  on  Block 


OPERATOR  NAME:  op4 
pentns 

(<tt>4oo  location-model) 

(■cfrom-loo  location-model) 

(<monkry>  monloey-model) 

(<Mock>  block-model) 
praoond* 

(connected  <to-loo  <from-k>») 

(connected  <from-loo  <to-loo) 

(rnoolcey -at- location  <monkey>  <from-lo») 
(block-at-location  <block>  <from-loo))) 
effects 

(del  (monkey-at-location  <monkey>  <from-loo)) 
(del  (block-  at-  location  <block>  <from-loo)) 

(add  (monkey-at-location  <monkey>  <to-loo)) 
(add  (block-at-loceion  <block>  <to-loo)) 
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Question  7 


□  QanOaa.  Qmw «— 


START  STATE 
(on-ceding  banana  hook) 

(blocks  location  block  loc4) 
(monkey-at-locadon  monkey  loc3) 
(connected  loc2  loc4) 

(connected  loc2  loc3) 

(connected  loci  ioc2) 

(connected  Ioo4  loc2) 

(connected  k>c3  loc2) 

(connected  loc2  loci) 

(under-hook  loci  hook) 

GOAL  STATE 

(holding-banana  banana  monkey) 


What  is  the  goal  of  probleml? 

1)  To  have  the  monkey  on  the  block 

2)  To  have  the  monkey  at  location  3 

3)  To  have  the  monkey  holding  tha  ha- 

nnnnm 

4)  None  of  the  above 


Question  8: 


□« — 


START  STATE 
(on-ceiling  banana  hook) 
(block-at-location  block  loc4) 
(monkey-at-locadon  monkey  ioc3) 
(connected  loc2  locd) 

(connected  loc2  loc3) 

(connected  loci  k>c2) 

(connected  loc4  loc2) 

(connected  loc3  loc2) 

(connected  loc2  loci) 

(under-hook  loci  hook) 

GOAL  STATE 

(holding-banana  banana  monkey) 


In  the  start  state  of  probleml  where  is  the  block? 

1)  Loci 

2)  Loc2 

3)  Loc3 

4)  Nona  of  tha  above 
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Question  9: 


START  STATE 
(on-ceiling  banana  hook) 
(Uock-at-location  block  k>c4) 
(monkey-at-location  monkey  k>c3) 
(connected  loc2  loc4) 

(connected  loc2  loc3) 

(connected  loci  loc2) 

(connected  loc4  loc2) 

(connected  loc3  k>c2) 

(connected  loc2  loci) 

(under -book  loci  hook) 

GOAL STATE 

(holding-banana  banana  monkey) 


In  the  start  state  of  probleml  where  is  the  monkey? 

1)  Loci 

2)  Loc2 

3)  Loc3 

4)  Loc4 


Question  10: 


This  domain  is  for? 

1 )  Stacking  Blocks 

2 )  Moving  Packages 

3)  Monkey  getting  bananas 


E-ll 


Appendix  F  -  Code  for  Medium  Size 

Domain 


F-l 


(aefladddte  atWI) 

■  OuilillW  nnirl  *hltdOM»  a*iU40*4»)> 


( IM  IW>MI  1  mnl  «W1IWM»I ) 

— «  ||«  aaallafale  i mil  lailrla  «fcill40*4»)  I 
(add  (la  taalldila  mnl  <d  blc*0*«»)) 

M  it  -n-'-  ddWIIb  atatola40*4»)  I ) ) ) 


<viaa«2Se>  «jrlU«2S*>.|l 
a-catta  afciUttSOI) 
>laa43Sd»  atabla«3Sd»»)) 


<(.ckiU730  <killl  {aHdtHfe.  drlll-dU) 
indiHlfUr  cablall) 


(aal  (la  — Udila  mnl-talfr  «K1U7SM 
(la  aaalldlla  mnl  ad-MCHt») 

(oa-caMa  ad-tdt75«>  «t*WKblll 

^daT'la  — lldila  mil  lulild  •AiUTStol) 
(dal  (la  awlliWa  Will  ad-fcit7S«»)) 

(dal  111  rdda  «*-Me75*»  udilallMfa)) 

(add  (haldlad-cool  4UCIM>  AUUSWIIII 


■  <(adkill*U»  *ul)  (<rlaa*U>  viaa) 
( rtdllatrtto.  eaMal)) 


:  (la  aralldila-tabla  ndrtUdli*)  ■ 
(la  m**  hnliHnQTlaa  «taa*13») 
|c  -dill  aviaadU>  atflaWMall) 


((Ml  (la  awlldila-tadla  «lrtll«U>>) 

(dal  (MdU  aviaadU*  icfladWh)) 

(add  [Ida  ilatlna  artaadlH  «klU*U»))))) 


HaalUrhaBlII 
a  (la  aralUMa  parr  «*ar£UJ7M) 
I  ((add  (elaaa  -frturrx  I  > ) ) 


((«vtaaU12>  viaa)  (apaaeUtt*  part) 
|— tatolfc))) 
da  II" '-*‘*7  frt4J13»  avlaatllM) 


( l**.!  (holdtaq  <wre4312>  aiaaUlt ) ) 

(Ml  (ro-<d>l*  OKC43L2>  <eahladM3>H 
(add  (ia-aaailadla-pare  *parc*J12»ll 
(add  (la  m*¥  Mdl«* «laa  aviaa4312 >1)1)1 

:  <MU-«d.a»-Moe-*iU 


(<aald»  pKC-aldal  (alecao  aadar^ypa)  (alocy»  wadjar-cypa) 
;  ignrlHIT-  «KI  («*±U9S1S»  drill)  (aviaa9515»  viaa) 
<-frtSSlS»  part)  (<a-d»SlS>  dad-drill))) 


(aad  (rlai  •*art*S15»l 

(bttldlaa  HttddlS*  «»iaa9SlS»l 
inaa  ilarira  atadflS  «killdS15>l 
daldbv^ool  <a  d9S15>  alrill9S15>l ) ) 

(adtaeca 

|  (dal  (elaaa  «pareK15>)> 

(add  ly-1—)"  <apocl2«47>  oida  <iocx>  <locy») ) 
(add  (liaa  liaia  -parc9il5>l ) 

(add  Otta-fec  aafC13M?»  apaRd515>) )  I ) ) 

:  (klU-«td>-aaaaWa:-aiaiad-<kiU 


L(  man'll  - - it-  (daoladddda  Kola)  (aklll337(>  Mill) 

(ak  ffldiar  — n — I  (<a-f-<flJ7*»  aaaife-auead-Milll 
(<viaaM02>  via.)  (fttaiOS*  pare)  <a*lM»  part-aida) 

(<low>  aatarT^pa)  (<locy»  nafr-eypal  (f»e2t»3>  npoe))) 


(aad  (ooe-looadcn  lam)  in')*  aalrlav  <loaa  <laey>) 
(elaaa  frtSdOS*) 

(aaearlal-of  <gaxc24«S»  taraaal 
■“-| - a#-l-<fij?«»  ad» 


(haa  Jaalna  oiaddlfe  akilU37C>) 

ItiM  dint  T~-*»-  i - - 

aaaldUa-eaal  <a-l- cBJ7*»  «ririU2J7fc»l )  1 
(•£ftCCS 

( (<u  {^oc-bndn  apx24l3>  <loc»  <Xocy>) ) 

(dal  (olaai<parc2«0S») ) 

(dal  (tiaa  emr  aapoe24i3>  «part340S»l) 

(add  Otola-lrxariai  di-lT*!— -  <alda»  adaptto  «i>  aleao  <locy»l) 
dM-duaxa  aadlos>)) 

(add  (fur  tola  dwladldlh  <part340S») >))) 


r  AW-adcte-adac-ikin 


'(avlao  viaa)  ( 


pa)  (afcelaa  tela)  («fcUl»  *1UI 
te»  pare)  (aaidaa  pare-alda) 
ml  (rloey»  oadiar-cypa)  (aappo  «dl 
«t-d>  wlae-dM.il) II 


■  ((otaaUMa  viaa)  (• 
I  KdalaOdda  eaUa))) 


frr17Tti) 
rlaa  aviaaUM*))) 


eUM»l) 

aataatfMM) 

LaaUJfl)))) 


>  <Locr>  <loey>) 


«rlaaa) 
h»  aklU» 


( (dal  (^ot-looaeiao  « 
(dal  (elaaa  fa  tv)) 


<dtill»l)) 

go  aaldo  also»  alee**)) 


-I—  Mid  alia  -ay*-,  -aO  aleao  aleey») ) 
foil  (add  (haa-dnla  dmlf  fro))))) 


((adrtlMdM*  dfelU)  (ad  dtdtb  <fclU-tit) 


drill 

tip  haul  («kdll4M*>  drill)  «up4M4»  cap) 

I  imiatMti  aiaa)  (aida  pac-adda)  (afcpd»  ma*ar-cypa) 
(4*  ■■Car  cud  tilodP  mal  (<iocy>  wadar  rypal 

(-dala»  tala)  (paaclWti  put))) 


Mad  (aUaa 


H 


a) 

IhnMInt  anal  atetilWte) 

Itakdag  dadWfc  <*laa4*M» 

I  «rlaa*M4>  «kU14M4» 

»))) 


•  ate  <1«B>  <looy>) 


MU 

<adi  (1«  rapal  <aptm>  dnla 
(add  (tadan  aactfM»)) 

(add  Oaa-cap  <a*tm>  aacUMkiim 


<locy»l) 
<iocy» ) 


(laa  Quid  <g013>  <tart30tt>) 

Oaldlaa  coal  inaarfO»l>  adrilU0M» 
ll>l>1**,*1T  aadOtt>  <vlaa60ft>) 

(laa  darioa  <viaS«l>  afeillSMl>)' 
Paa  aula  <del»  «**rtJO»l») ) ) 


((dal 

(dal 

(dal 

«tel 

(add 

(add 

(add 


(claa  «tartSOdl>)  j 

Paa  fluid  dSOtb  <partS091>) ) 

(laa  tola  «twla»  «parc30*l») 
llanad  iwanni  data  <a(r)a»  dp 
Paa  hurra  <partS09i»| ) 

aa£10J>  <jactSO»l») ) ) )  > 


«te  aloaa  <loey»>) 


<Locy»> 


( (afcpcte  aadaa  qpa)  (ate  nadar  rypal  (dnla»dl»  taolal 
(afcllU«3t>  drill)  Mo-tt-rKIM*  oU-teld-dtUl) 

(aalda  pacc-dda)  (<loca»  malar  rypa)  «lacy»  mafeac-typa) 
WddBda  a«>  («viaadt3d>  viaal  (<5*rt*t3C»  pace) 
(<WH»  Quid))) 


r  CktU-alttteailakiU 


dklll-aidt-o-b 


((— gla  mafcar  cypa)  (a 

<a*UlS007>  (kill)  (dacCS04]>  pact) 
(aviaaMU*  «iaa)  (  «*6(>ddi  macatml  « 
(adaca  radar  r^pal  (a*  madar  ratal  (<1 
(<locy»  aatar  qpa)  (dola  tela) ) ) 


(and  (hala-locadoa  del»  -ml rla  data  «i>  <l«gp  <locy>) 
( U  aarac  «*SQ4*>  ate) 

Iclaaa  narrt043>) 

(teUdaveoal  «*MM»  «kllisa27» 

) 

(I 
(] 


aiiidUdfc  adrlUS407>) 
))) 


(ad  (elaaa  <cartM38>) 

(dot-location  <npocM3t»  aatrta  <lora>  <locy» 
(laa  Quirt  <KMb  <part*«31>l 
(haldlag-BBal  adrllMUai 

•) 


Paa  darioa  «via*M3d>  «kiU(dM») 
(ha  pm  opociUb  <p«rc*W*>) ) ) 


<locy» ) 


((dal  (elaaa  -<tartM3t» ) 

(dal  (aoe-locaeii 

(dal  (haa-Quid  <f6Mt>  «part&VR>) ) 

(dal  Oaa-aac  <C«SBt>  «*artM3*»l  I 

(add  OBla-laaalsa  dola<MS>  «alda  date  ate  <locx>  <iocy» ) 

(adl  Paa  da  i  a  apart**)**! > 

(add  Paa  hola  dnlaCMS*  .partM3b) ) )  > ) 


((dal 

(<tel  (rlaa  dactSMlal) 

«tel  paa  hnla  doia  aaceSMM) 

(add  U-aaaatea  add>  ala  ate 
(adl  Paa  Mara  <pactS04J»l) 


hatj  ia-t  («tabla7007>  cabla) 
))) 

<±and_*a?007»> 
<aad70Q7>) 

■caw 7 007 >  <tabla7007>) ) ) 


(Ma»  «arc-aiaa>  (—arc704«>  pare) 


.  -vkTOM*) 

lul  I 


ae^m.|g  «tn 

>  i  ■— iiiiTW',  -»m—  mOdaM  (<caU«Um>  Mia) 
(M-tltUTIl*  ibrlll-biE) )  > 

(and  (aac-««r-tor  — dlll37tl>  dUUflti 

(r»-i^iH««lr  firm  Mr  In  rlH  <adilU7*l»* 

[!■  rd)l~  44tcl)7ll>  <caHtUW>l 
II  a  anil  ml  a  nnnl  M-taitU7tl>)  I ) 


({dai  lua  taMtoddiU  fctt  Until  4dUlHtt>H 
1^1  na  I  dll  a  M-taiclJ7U>  <MUU7»1»)) 

(«M  <haldbv-*lU-Mc-i»*dU  aMdeUTll*  «U1UW»I)1)I 

LMt  rl—  In  Mill 

pmmt  -iih«b  —aiMl  (<*i*«U»12>  *iaa) 

WablaUltf*  MU») 


lad  (noe-baldlag-alaa-la-aiU  aUUMW 
fna  rbda  oliaUnb  aablaUHU 
[la  aun  bnl — If  - - — •»<-" 


_ , _ l  MUUftt»> 

(dal  (aa-cabta  <vtaallM2»  <MUU*12»t) 

Ml  *»!*»■* l— 1 ‘a  bill  «*iaaU*12>  aUUlMW I ) ) ) 

lamia  rlaa  fi  n  MU 

(pm  ( imaab  dlllujadlaal  I<riaaua22>  dial 
(■eMLaUM2»  cabtolll 

alll  <viaal3*32»  — I1113a32>) 

Hi  aiat  laibllm  Tin  olaalWMI) 


lldai  Qaldtod  alaa  la  adl)  <rt—13*Z2>  aUUMMI 
(aM  faa  laalallial  i~laa  In  aril)  aUUWb)) 
ladt  Mtabla  MaalMBa  «aMaUS&»»» 


KadllUWla  dUlauadM  Id4ttum>  drill -bit) 
[<tdteU«2>  cabUOl 


.  mdhrdlll-MMa  adll  d  WtUBI*  ailllMIW 
(la  adldli  «aal  «MU»U»))> 


((dal  [UiVMbl  lb  111  Nr  In  all[  ddltllflt>  aiHtMJWI 

Ml  Inal  jaliMaq  MU  tic  1 - 111  M1U3M1»II 

(adl  fa  nfi  Mbit UM1»  <MlaUMte>)))) 

alll  adab-MU 

I ( HUM—  —lira  ~rr~'  IdddlMbWal  (—-*  am-*tll) 
WUimb  amlaajaMaal  IdabaW 
(«e-dUMJ»  Batat-*iU)  (MaalJim  alaa)  (MatlJHi*  pawl 
(-iT-  paaT  aldaf^  * 


(aaciv-te -adUUMb* 

alii  <t-dUM3»  MUUMte) 
QaiiHaa  -j— — *n  aril!  <viaaDMl»  — 111W40) 
niiMwmf  datUW>  «aiaaU*4J»> 
mM  a  -  <a«UW>  *aarcUMl» 

(la  awllibla  rani  «*-dlJ»43») ) ) 


( >,wi  (^oe-locadan  <nl<M  <locy>t  > 

(dal  (elaa  batUHWI 

/«wi  dM-apna  <aocliNb  ^accMM3>H 

(aM  (haMlaaaeloa  MM  «b  loo»  -doqrW) 

Ml  flaw  him  4iRUIO») 

(adl  Qaa-bala  dnla!394fe  dactUMbll))) 


it  («loey» 


pc -drill 


KMa  part-aidal  (<loc - 

(<acocU*7t>  vac)  MM  alll -drill) 

(MlUNOa  ainim  Mblaa)  (odUMb  bBbddUl 

(«p— cUHO»  pare)  (aaiaaUMO*  alaa))) 


l  (« -  - - 

iaac-vxr-for  MllUMOa  -oaMI 
HUbrdlllblB-la  am  <a-dU»«0»  Mlllbbl 
(adlpalMHUl  <*iaaUMO>  <m1111J9*0>) 
llaldbl  ^artUHOa  aaiaal  IWb) 

(la  aaailthla-taal  <a-dUKO»>  •  I 


((dal  <ela« —aclllWOMI 

1^  (^oc-locad—  <«JOCUr7*>  fldfa  <looo  <locy>) ) 
(adl  (laa  Harra  «parel3»«b»» 

(■U  lba-«K  <^ocl377t>  <parel39M>)  1)11 

pue-adlllap-duccar-ia-aim 

(paaaa  ( «MU1400»»  cabUI  («ajsl40ia»  «dllina_cuctar> 
(— 11114010»  amiaajaddna) ) ) 


M  (aac-M-te  — 11114010>  allliio) 

imj^aaaaaa»(dla  — jci4Qio>  <tablal4009>) 
Uac-MdiiigMlll-bic-ib-am.  Mlll«010>l  > ) 


dial  c*  ii  iiy—  imp  djldlb  <cabl*14009>) ) 

(dal  ooc-inldladMm-blc-iaMU  —mi4010»)> 

(adl  «■»»«■  naiai  m  all)  — jb14010*  — 1U14010*) ) ) ) ) 


U<cablal401fl>  Mia)  MUlMOlIb  aUlisaaachina) 
Mjd.401t>  aUlIngjirnar))) 


(ad  bamiaw-amar  adU  MJ3140M  «*tlH40ia»l 
(i>«  laiiiMibl  rlaa  In  alll  —iUMOUa))) 


(< _ 

((dal  P^IHiy— arappaalll  aidlOlb  — 11114011>) ) 
(add  ft-piiiM  nataa  '■>  (dila  —_cl40ia>  <cablal40ll>) ) 
(adl  Me-bBldtad-drill-ble-ln-adll  M1U4011» ) ) ) ) 

:  adJl-M* 


(MiaaS*  part-alaa)  (CHl  alll-<biU) 
f  ■milffrr  amiaa  aarblnal  (<viaal4023>  vlaa) 
(M»  PBC-Mdal  («XMK<1>  parc-aurfana  ciaallrlnn) 
(«d»  iiiaaaalm  aiiijal  Miaal*  parc-aixa) 
(aarCl40SS>  pad)  (Mjel«n3>  aOlinajaicxar) ) ) 


Idea  -VKCl402}>  <a0>  <oad>) 
i  -VKCl«Cai>  MM  «ial>l 
(aac-tv-te  a)llHB1>  <a-d»l 
MUlag  qaecaa  in  alll  mj=14<8J>  «adlU4flaj»l 
(baldM-rlaa-lDMU  <rlaal4021>  —1111402 1>) 
IholdlaB  — arcl402}>  «*laal4caj» ) ) 


— «ecl4021»  <ad>  «oaadl>) ) 
<alial>) ) 
<par«14C23>  <ad>  oouM> 
))))) 


( ( iiddallinti  tabU)  (<*iaaU108»  rlaa) 
(MdaaMUSa  pbaart ) ) 

'lad  (aa-Mla  <*Ual410S»  <MU14104» 

(la  amr  hnliUnp  alaa  <riaal410S»)  I ) 

((dal  (m  Mia  oiaaHWa  <rab)al4104») ) 

(add  (laililll  rln  In  (il—r  <viaal410S»  Ml4Barl410S») )))) 


aaiaallin*  «Ua— 1410*» 


(la* 


-Mdkaa-oia 


im.  a 

tm 


I  »>■  t»pl« 


r  «toH1l»  tlafUMWI 

'ill)) 


imd  fMM  imUrlnn  «mrMU9»  «■*! 

«kcM11S»  j**' —  !»<■»> 
miri — r-— -1— UUO) 


IB  all  fcraiUti 

_  (1MI»  MU-Suil  <«ajlueaO>  aill*P*laHI 

(nCH^te 

(MM  irtllWBOfc 

(aril  (HC-vlor  ^Uiaacfr  ^Uisol)))) 


— *•  -n  HI  fm — y*">~g 

([m  J,  BlU-cfclU)  (— t1UW»»»<lllnajH«to1») 
f.|  i  ■  ■  4.  «dU303C3>  ■  an 

imttmcCM  KcMl  (mc-itf-tar  <^HKIM3> 

(«c-o-toc  «uu<ae>  aruiiiw)))) 
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Appendix  G  -  Learning  Study  Domains 


G*1 


Hiking  World  Description 

In  »hi«  a  person  is  hiking  with  a  backpack  from  one  campground  to  another 

campground.  The  objects  in  this  domain  are:  backpack,  person,  and  campgrounds. 
Relations  between  objects  are  as  follows:  1)  a  person  can  hold  a  backpack;  2)  a  person  can 
be  at  a  campground;  3)  a  backpack  can  be  at  a  campground;  and  4)  two  campgrounds  can 
be  connected  to  each  other.  The  actions  that  are  performed  in  this  domain  are:  1)  a  person 
can  pfcimp  a  backpack;  2)  a  person  can  put  down  a  backpack;  and  3)  a  person  can  move 
from  one  campground  to  another  connected  campground. 

T»«ir-  Define  an  initial  state,  goal  state,  and  domain.  Have  the  system  find  the  plan  needed 
to  go  from  initial  to  goal  state  of  the  following  problems. 

Problem  1:  For  the  initial  state  create  two  campgrounds  (campl,  camp2),  a  person,  and  a 
hyirpa^if  Campl  and  camp2  are  connected  together.  Have  the  person  and  backpack  start  at 
campl.  The  goal  is  to  have  the  backpack  sitting  at  camp2. 

Problem  2:  Use  the  work  from  the  previous  problem.  For  the  initial  state  create  three 
campgrounds  (campl,  camp2,  camp3),  a  person,  and  a  backpack.  Campl  is  connected  to 
camp2  and  camp2  is  connected  to  camp3.  Have  the  person  start  at  campl  and  the  backpack 
at  campl.  This  time  the  goal  stale  is  to  have  the  backpack  sitting  at  ca mp3. 


G-2 


Loading  Truck  World  Description 

In  this  domain  a  person  will  be  loading  a  truck  with  packages  from  a  warehouse.  The 
objects  in  this  domain  are:  truck,  warehouse,  person  and  packages.  The  relations  that  exist 
between  the  objects  axe:  1)  a  package  can  be  on  the  truck;  2)  a  package  can  be  in  the 
warehouse;  3)  a  person  can  be  on  the  truck;  and  4)  a  person  can  be  in  the  warehouse.  The 
yrinn«  that  ran  be  [performed  in  this  domain  are:  1)  pick  a  package  up  at  the  warehouse;  2) 
Take  a  package  from  die  warehouse  to  the  truck;  3)  put  a  package  in  the  truck;  and  4)  Go 
from  the  truck  back  to  the  warehouse. 

Tagir*  Define  an  initial  state,  goal  state,  and  domain.  Have  the  system  find  the  plan  needed 
to  go  from  initial  to  goal  state  of  the  following  problems. 

Problem  1:  For  the  initial  state  create  a  truck,  a  package,  a  warehouse,  and  a  person.  The 
person  is  at  the  warehouse  along  with  the  package.  The  goal  is  to  have  the  package  loaded 
onto  the  truck. 

Problem  2:  Use  the  work  from  the  previous  problem.  For  the  initial  state  this  time  have  the 
person  start  out  at  the  truck  and  the  package  at  the  warehouse.  The  goal  is  again  to  have  the 
package  loaded  onto  the  truck. 


G*3 


Robot  Picking  Tulip  Description 

In  this  domain  a  robot  is  going  around  to  different  locations  and  picking  up  tulips.  The 
objects  in  are:  robot  with  a  basket,  tulips,  and  locations.  Relations  between 

objects  are  as  follows:  1)  a  robot  with  a  basket  can  hold  a  tulip  in  the  basket;  2)  a  robot  with 
a  basket  can  be  at  a  location;  3)  a  tulip  can  be  at  a  location;  and  4)  two  locations  can  be 
connected  to  each  other.  The  actions  that  are  performed  in  this  domain  are:  1)  a  robot  with  a 
bad n*  can  a  tulip  and  put  it  in  a  basket;  and  2)  a  robot  can  move  from  one  location 

to  another  connected  location. 

Ta«ir-  Define  an  initial  state,  goal  state,  and  domain.  Have  the  system  find  the  plan  needed 
to  go  from  initial  to  goal  state  cf  the  following  problems. 


Problem  1:  For  the  initial  state  create  two  location  (Loci,  Loc2),  a  robot,  and  a  tulip.  Loci 
and  Loc2  are  connected  together.  Have  the  robot  start  at  Loci  and  the  tulip  start  at  Loc2. 
The  goal  is  to  have  the  robot  hold  the  tulip  in  its  basket 

Problem  2:  Use  the  work  from  the  previous  problem.  For  the  initial  state  create  three 
locations  (Loci,  Loc2,  Loc3),  a  robot,  and  two  tulips.  Loci  is  connected  to  Loc2  and  Loc2 
is  connected  to  Loc3.  Have  the  robot  at  Loci,  a  tulip  at  Loc2,  and  a  tulip  at  Loc3.  This  time 
die  goal  state  is  to  have  the  robot  holding  two  tulips  in  its  basket 
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